2023-09-12T14:40:41.492 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-12T14:40:41.493 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395049 --owner scheduled_pdonnell@teuthology --verbose --name pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota} -- /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395049/orig.config.yaml 2023-09-12T14:40:41.536 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-12T14:40:41.627 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395049 branch: wip-batrick-testing-20230912.122437 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota} email: pdonnell@redhat.com first_in_suite: false job_id: '7395049' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-batrick-testing-20230912.122437 ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: client: pid file: '' mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 2110e00747b31a0d73768c9e5229da18b26b8aa0 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 2110e00747b31a0d73768c9e5229da18b26b8aa0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20230912.122437 sha1: 3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 9684 sha1: 2110e00747b31a0d73768c9e5229da18b26b8aa0 sleep_before_teardown: 0 subset: 236/256 suite: fs suite_branch: wip-batrick-testing-20230912.122437 suite_path: /home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph.git suite_sha1: 3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi131.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDimC56/hFFPRUSSnJw57ISHBvV1b41xUIVLsFaU/Q5jjFtRty1ECTHJAz1lZsZYADp2Hs6BIyUbsb23UEm5CDs= smithi187.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPlylNRPAXfYR2xFtC39/4ZS32+rbGLFQP0faK3NmYDP8iUE2PbEIiYiJxboXIcK55ZSOLyk2idPQ2YJ9Br2zsM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_quota teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-09-12_14:07:50 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.1394 2023-09-12T14:40:41.627 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa; will attempt to use it 2023-09-12T14:40:41.628 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks 2023-09-12T14:40:41.628 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-09-12T14:40:41.630 INFO:teuthology.task.internal:Checking packages... 2023-09-12T14:40:41.653 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '2110e00747b31a0d73768c9e5229da18b26b8aa0' 2023-09-12T14:40:41.653 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-12T14:40:41.653 INFO:teuthology.packaging:ref: None 2023-09-12T14:40:41.654 INFO:teuthology.packaging:tag: None 2023-09-12T14:40:41.654 INFO:teuthology.packaging:branch: wip-batrick-testing-20230912.122437 2023-09-12T14:40:41.654 INFO:teuthology.packaging:sha1: 2110e00747b31a0d73768c9e5229da18b26b8aa0 2023-09-12T14:40:41.655 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-batrick-testing-20230912.122437 2023-09-12T14:40:41.871 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6088-g2110e007-1jammy 2023-09-12T14:40:41.872 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-09-12T14:40:41.878 INFO:teuthology.task.internal:no buildpackages task found 2023-09-12T14:40:41.878 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-09-12T14:40:41.893 INFO:teuthology.task.internal:Saving configuration 2023-09-12T14:40:41.909 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-09-12T14:40:41.918 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-12T14:40:41.944 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi131.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395049', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-12 14:35:09.858157', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILr6Fk3U1Ev/IXB6dmPjIabjS/KBdRRSjiEgBrg2xkb8'} 2023-09-12T14:40:41.970 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi187.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395049', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-12 14:35:09.856194', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIA8KntlocJQMi/AXJowG0JUaJE3NjfbdQQV0sg7U9Duk'} 2023-09-12T14:40:41.970 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-09-12T14:40:41.979 INFO:teuthology.task.internal:roles: ubuntu@smithi131.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-09-12T14:40:41.980 INFO:teuthology.task.internal:roles: ubuntu@smithi187.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-09-12T14:40:41.980 INFO:teuthology.run_tasks:Running task console_log... 2023-09-12T14:40:42.056 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f33faa8db80>, signals=[15]) 2023-09-12T14:40:42.057 INFO:teuthology.run_tasks:Running task internal.connect... 2023-09-12T14:40:42.066 INFO:teuthology.task.internal:Opening connections... 2023-09-12T14:40:42.066 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi131.front.sepia.ceph.com 2023-09-12T14:40:42.068 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-12T14:40:42.145 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi187.front.sepia.ceph.com 2023-09-12T14:40:42.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-12T14:40:42.219 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-09-12T14:40:42.227 DEBUG:teuthology.orchestra.run.smithi131:> uname -m 2023-09-12T14:40:42.232 INFO:teuthology.orchestra.run.smithi131.stdout:x86_64 2023-09-12T14:40:42.233 DEBUG:teuthology.orchestra.run.smithi131:> cat /etc/os-release 2023-09-12T14:40:42.277 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-09-12T14:40:42.277 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="Ubuntu" 2023-09-12T14:40:42.278 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="22.04" 2023-09-12T14:40:42.278 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-09-12T14:40:42.278 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_CODENAME=jammy 2023-09-12T14:40:42.278 INFO:teuthology.orchestra.run.smithi131.stdout:ID=ubuntu 2023-09-12T14:40:42.278 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE=debian 2023-09-12T14:40:42.279 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://www.ubuntu.com/" 2023-09-12T14:40:42.279 INFO:teuthology.orchestra.run.smithi131.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-09-12T14:40:42.279 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-09-12T14:40:42.279 INFO:teuthology.orchestra.run.smithi131.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-09-12T14:40:42.279 INFO:teuthology.orchestra.run.smithi131.stdout:UBUNTU_CODENAME=jammy 2023-09-12T14:40:42.280 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2023-09-12T14:40:42.299 DEBUG:teuthology.orchestra.run.smithi187:> uname -m 2023-09-12T14:40:42.304 INFO:teuthology.orchestra.run.smithi187.stdout:x86_64 2023-09-12T14:40:42.304 DEBUG:teuthology.orchestra.run.smithi187:> cat /etc/os-release 2023-09-12T14:40:42.349 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-09-12T14:40:42.349 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="Ubuntu" 2023-09-12T14:40:42.349 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="22.04" 2023-09-12T14:40:42.349 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-09-12T14:40:42.350 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_CODENAME=jammy 2023-09-12T14:40:42.350 INFO:teuthology.orchestra.run.smithi187.stdout:ID=ubuntu 2023-09-12T14:40:42.350 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE=debian 2023-09-12T14:40:42.350 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://www.ubuntu.com/" 2023-09-12T14:40:42.350 INFO:teuthology.orchestra.run.smithi187.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-09-12T14:40:42.351 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-09-12T14:40:42.351 INFO:teuthology.orchestra.run.smithi187.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-09-12T14:40:42.351 INFO:teuthology.orchestra.run.smithi187.stdout:UBUNTU_CODENAME=jammy 2023-09-12T14:40:42.351 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server 2023-09-12T14:40:42.374 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-09-12T14:40:42.384 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-09-12T14:40:42.392 INFO:teuthology.task.internal:Checking for old test directory... 2023-09-12T14:40:42.393 DEBUG:teuthology.orchestra.run.smithi131:> test '!' -e /home/ubuntu/cephtest 2023-09-12T14:40:42.395 DEBUG:teuthology.orchestra.run.smithi187:> test '!' -e /home/ubuntu/cephtest 2023-09-12T14:40:42.400 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-09-12T14:40:42.407 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-09-12T14:40:42.408 DEBUG:teuthology.orchestra.run.smithi131:> test -z $(ls -A /var/lib/ceph) 2023-09-12T14:40:42.443 DEBUG:teuthology.orchestra.run.smithi187:> test -z $(ls -A /var/lib/ceph) 2023-09-12T14:40:42.500 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-09-12T14:40:42.551 INFO:teuthology.run_tasks:Running task kernel... 2023-09-12T14:40:42.565 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-09-12T14:40:42.565 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-09-12T14:40:42.566 DEBUG:teuthology.orchestra.run.smithi131:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-12T14:40:42.567 DEBUG:teuthology.orchestra.run.smithi187:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-12T14:40:42.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T14:40:42.571 DEBUG:teuthology.orchestra.run.smithi131:> uname -r 2023-09-12T14:40:42.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T14:40:42.573 DEBUG:teuthology.orchestra.run.smithi187:> uname -r 2023-09-12T14:40:42.617 INFO:teuthology.orchestra.run.smithi131.stdout:5.15.0-56-generic 2023-09-12T14:40:42.618 INFO:teuthology.task.kernel:Running kernel on smithi131: 5.15.0-56-generic 2023-09-12T14:40:42.618 DEBUG:teuthology.orchestra.run.smithi131:> sudo apt-get clean 2023-09-12T14:40:42.623 INFO:teuthology.orchestra.run.smithi187.stdout:5.15.0-56-generic 2023-09-12T14:40:42.624 INFO:teuthology.task.kernel:Running kernel on smithi187: 5.15.0-56-generic 2023-09-12T14:40:42.624 DEBUG:teuthology.orchestra.run.smithi187:> sudo apt-get clean 2023-09-12T14:40:42.753 DEBUG:teuthology.orchestra.run.smithi131:> sudo apt-get update 2023-09-12T14:40:42.775 DEBUG:teuthology.orchestra.run.smithi187:> sudo apt-get update 2023-09-12T14:40:42.886 INFO:teuthology.orchestra.run.smithi131.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-09-12T14:40:42.902 INFO:teuthology.orchestra.run.smithi131.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2023-09-12T14:40:42.918 INFO:teuthology.orchestra.run.smithi187.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2023-09-12T14:40:42.968 INFO:teuthology.orchestra.run.smithi131.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2023-09-12T14:40:43.002 INFO:teuthology.orchestra.run.smithi131.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2023-09-12T14:40:43.027 INFO:teuthology.orchestra.run.smithi187.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-09-12T14:40:43.103 INFO:teuthology.orchestra.run.smithi131.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [484 kB] 2023-09-12T14:40:43.109 INFO:teuthology.orchestra.run.smithi187.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2023-09-12T14:40:43.137 INFO:teuthology.orchestra.run.smithi187.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [764 kB] 2023-09-12T14:40:43.168 INFO:teuthology.orchestra.run.smithi131.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [972 kB] 2023-09-12T14:40:43.194 INFO:teuthology.orchestra.run.smithi131.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [223 kB] 2023-09-12T14:40:43.199 INFO:teuthology.orchestra.run.smithi131.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [15.6 kB] 2023-09-12T14:40:43.200 INFO:teuthology.orchestra.run.smithi131.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [30.3 kB] 2023-09-12T14:40:43.200 INFO:teuthology.orchestra.run.smithi131.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [843 kB] 2023-09-12T14:40:43.210 INFO:teuthology.orchestra.run.smithi131.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [136 kB] 2023-09-12T14:40:43.211 INFO:teuthology.orchestra.run.smithi131.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [536 B] 2023-09-12T14:40:43.211 INFO:teuthology.orchestra.run.smithi131.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [979 kB] 2023-09-12T14:40:43.220 INFO:teuthology.orchestra.run.smithi131.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [653 kB] 2023-09-12T14:40:43.223 INFO:teuthology.orchestra.run.smithi187.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [319 kB] 2023-09-12T14:40:43.228 INFO:teuthology.orchestra.run.smithi187.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [165 kB] 2023-09-12T14:40:43.230 INFO:teuthology.orchestra.run.smithi187.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.3 kB] 2023-09-12T14:40:43.230 INFO:teuthology.orchestra.run.smithi187.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [30.0 kB] 2023-09-12T14:40:43.231 INFO:teuthology.orchestra.run.smithi187.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [826 kB] 2023-09-12T14:40:43.247 INFO:teuthology.orchestra.run.smithi187.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [133 kB] 2023-09-12T14:40:43.249 INFO:teuthology.orchestra.run.smithi187.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [536 B] 2023-09-12T14:40:43.250 INFO:teuthology.orchestra.run.smithi187.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [557 kB] 2023-09-12T14:40:43.255 INFO:teuthology.orchestra.run.smithi131.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [213 kB] 2023-09-12T14:40:43.257 INFO:teuthology.orchestra.run.smithi187.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [781 kB] 2023-09-12T14:40:43.258 INFO:teuthology.orchestra.run.smithi131.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [21.8 kB] 2023-09-12T14:40:43.258 INFO:teuthology.orchestra.run.smithi131.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [41.6 kB] 2023-09-12T14:40:43.258 INFO:teuthology.orchestra.run.smithi131.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [3,888 B] 2023-09-12T14:40:43.259 INFO:teuthology.orchestra.run.smithi131.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [9,768 B] 2023-09-12T14:40:43.259 INFO:teuthology.orchestra.run.smithi131.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [476 B] 2023-09-12T14:40:43.260 INFO:teuthology.orchestra.run.smithi131.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [41.7 kB] 2023-09-12T14:40:43.260 INFO:teuthology.orchestra.run.smithi131.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [33.9 kB] 2023-09-12T14:40:43.260 INFO:teuthology.orchestra.run.smithi131.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [10.5 kB] 2023-09-12T14:40:43.263 INFO:teuthology.orchestra.run.smithi187.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [143 kB] 2023-09-12T14:40:43.264 INFO:teuthology.orchestra.run.smithi187.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.7 kB] 2023-09-12T14:40:43.274 INFO:teuthology.orchestra.run.smithi131.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2023-09-12T14:40:43.292 INFO:teuthology.orchestra.run.smithi131.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [13.4 kB] 2023-09-12T14:40:43.292 INFO:teuthology.orchestra.run.smithi131.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [24.3 kB] 2023-09-12T14:40:43.293 INFO:teuthology.orchestra.run.smithi131.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.4 kB] 2023-09-12T14:40:43.293 INFO:teuthology.orchestra.run.smithi131.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2023-09-12T14:40:43.295 INFO:teuthology.orchestra.run.smithi187.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,032 B] 2023-09-12T14:40:43.295 INFO:teuthology.orchestra.run.smithi187.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [36.5 kB] 2023-09-12T14:40:43.296 INFO:teuthology.orchestra.run.smithi187.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,060 B] 2023-09-12T14:40:43.296 INFO:teuthology.orchestra.run.smithi187.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2023-09-12T14:40:43.432 INFO:teuthology.orchestra.run.smithi131.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [319 kB] 2023-09-12T14:40:43.463 INFO:teuthology.orchestra.run.smithi187.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2023-09-12T14:40:43.609 INFO:teuthology.orchestra.run.smithi187.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [972 kB] 2023-09-12T14:40:43.687 INFO:teuthology.orchestra.run.smithi131.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [764 kB] 2023-09-12T14:40:43.827 INFO:teuthology.orchestra.run.smithi131.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [165 kB] 2023-09-12T14:40:43.848 INFO:teuthology.orchestra.run.smithi131.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.3 kB] 2023-09-12T14:40:43.849 INFO:teuthology.orchestra.run.smithi131.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [30.0 kB] 2023-09-12T14:40:43.853 INFO:teuthology.orchestra.run.smithi131.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [826 kB] 2023-09-12T14:40:43.866 INFO:teuthology.orchestra.run.smithi187.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [484 kB] 2023-09-12T14:40:43.908 INFO:teuthology.orchestra.run.smithi187.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [223 kB] 2023-09-12T14:40:43.911 INFO:teuthology.orchestra.run.smithi131.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [133 kB] 2023-09-12T14:40:43.920 INFO:teuthology.orchestra.run.smithi131.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [536 B] 2023-09-12T14:40:43.920 INFO:teuthology.orchestra.run.smithi131.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [781 kB] 2023-09-12T14:40:43.923 INFO:teuthology.orchestra.run.smithi187.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [15.6 kB] 2023-09-12T14:40:43.924 INFO:teuthology.orchestra.run.smithi187.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [843 kB] 2023-09-12T14:40:43.960 INFO:teuthology.orchestra.run.smithi131.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [557 kB] 2023-09-12T14:40:43.974 INFO:teuthology.orchestra.run.smithi187.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [30.3 kB] 2023-09-12T14:40:43.975 INFO:teuthology.orchestra.run.smithi187.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [136 kB] 2023-09-12T14:40:43.981 INFO:teuthology.orchestra.run.smithi187.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [536 B] 2023-09-12T14:40:44.143 INFO:teuthology.orchestra.run.smithi131.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [143 kB] 2023-09-12T14:40:44.145 INFO:teuthology.orchestra.run.smithi187.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [653 kB] 2023-09-12T14:40:44.523 INFO:teuthology.orchestra.run.smithi131.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.7 kB] 2023-09-12T14:40:44.530 INFO:teuthology.orchestra.run.smithi131.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [36.5 kB] 2023-09-12T14:40:44.545 INFO:teuthology.orchestra.run.smithi131.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,032 B] 2023-09-12T14:40:44.546 INFO:teuthology.orchestra.run.smithi131.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,060 B] 2023-09-12T14:40:44.549 INFO:teuthology.orchestra.run.smithi131.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2023-09-12T14:40:44.716 INFO:teuthology.orchestra.run.smithi187.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [979 kB] 2023-09-12T14:40:44.825 INFO:teuthology.orchestra.run.smithi187.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [213 kB] 2023-09-12T14:40:44.844 INFO:teuthology.orchestra.run.smithi187.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [21.8 kB] 2023-09-12T14:40:44.846 INFO:teuthology.orchestra.run.smithi187.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [3,888 B] 2023-09-12T14:40:44.847 INFO:teuthology.orchestra.run.smithi187.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [41.6 kB] 2023-09-12T14:40:44.850 INFO:teuthology.orchestra.run.smithi187.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [9,768 B] 2023-09-12T14:40:44.851 INFO:teuthology.orchestra.run.smithi187.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [476 B] 2023-09-12T14:40:44.851 INFO:teuthology.orchestra.run.smithi187.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [33.9 kB] 2023-09-12T14:40:44.854 INFO:teuthology.orchestra.run.smithi187.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [41.7 kB] 2023-09-12T14:40:44.857 INFO:teuthology.orchestra.run.smithi187.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [10.5 kB] 2023-09-12T14:40:45.021 INFO:teuthology.orchestra.run.smithi187.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2023-09-12T14:40:45.104 INFO:teuthology.orchestra.run.smithi187.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [13.4 kB] 2023-09-12T14:40:45.109 INFO:teuthology.orchestra.run.smithi187.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [24.3 kB] 2023-09-12T14:40:45.228 INFO:teuthology.orchestra.run.smithi187.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.4 kB] 2023-09-12T14:40:45.279 INFO:teuthology.orchestra.run.smithi187.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2023-09-12T14:40:49.084 INFO:teuthology.orchestra.run.smithi131.stdout:Fetched 8,895 kB in 2s (5,147 kB/s) 2023-09-12T14:40:49.629 INFO:teuthology.orchestra.run.smithi187.stdout:Fetched 8,895 kB in 2s (3,654 kB/s) 2023-09-12T14:40:50.337 INFO:teuthology.orchestra.run.smithi131.stdout:Reading package lists... 2023-09-12T14:40:50.354 DEBUG:teuthology.orchestra.run.smithi131:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-09-12T14:40:50.412 INFO:teuthology.orchestra.run.smithi131.stdout:Reading package lists... 2023-09-12T14:40:50.575 INFO:teuthology.orchestra.run.smithi131.stdout:Building dependency tree... 2023-09-12T14:40:50.576 INFO:teuthology.orchestra.run.smithi131.stdout:Reading state information... 2023-09-12T14:40:50.713 INFO:teuthology.orchestra.run.smithi131.stdout:The following packages were automatically installed and are no longer required: 2023-09-12T14:40:50.713 INFO:teuthology.orchestra.run.smithi131.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-09-12T14:40:50.713 INFO:teuthology.orchestra.run.smithi131.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-09-12T14:40:50.714 INFO:teuthology.orchestra.run.smithi131.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-09-12T14:40:50.714 INFO:teuthology.orchestra.run.smithi131.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2023-09-12T14:40:50.714 INFO:teuthology.orchestra.run.smithi131.stdout: python2.7-minimal 2023-09-12T14:40:50.715 INFO:teuthology.orchestra.run.smithi131.stdout:Use 'sudo apt autoremove' to remove them. 2023-09-12T14:40:50.715 INFO:teuthology.orchestra.run.smithi131.stdout:The following additional packages will be installed: 2023-09-12T14:40:50.715 INFO:teuthology.orchestra.run.smithi131.stdout: linux-generic linux-headers-5.15.0-83 linux-headers-5.15.0-83-generic 2023-09-12T14:40:50.716 INFO:teuthology.orchestra.run.smithi131.stdout: linux-headers-generic linux-image-5.15.0-83-generic 2023-09-12T14:40:50.716 INFO:teuthology.orchestra.run.smithi131.stdout: linux-modules-5.15.0-83-generic linux-modules-extra-5.15.0-83-generic 2023-09-12T14:40:50.716 INFO:teuthology.orchestra.run.smithi131.stdout:Suggested packages: 2023-09-12T14:40:50.716 INFO:teuthology.orchestra.run.smithi131.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2023-09-12T14:40:50.717 INFO:teuthology.orchestra.run.smithi131.stdout:Recommended packages: 2023-09-12T14:40:50.717 INFO:teuthology.orchestra.run.smithi131.stdout: thermald 2023-09-12T14:40:50.761 INFO:teuthology.orchestra.run.smithi131.stdout:The following NEW packages will be installed: 2023-09-12T14:40:50.761 INFO:teuthology.orchestra.run.smithi131.stdout: linux-headers-5.15.0-83 linux-headers-5.15.0-83-generic 2023-09-12T14:40:50.761 INFO:teuthology.orchestra.run.smithi131.stdout: linux-image-5.15.0-83-generic linux-modules-5.15.0-83-generic 2023-09-12T14:40:50.762 INFO:teuthology.orchestra.run.smithi131.stdout: linux-modules-extra-5.15.0-83-generic 2023-09-12T14:40:50.762 INFO:teuthology.orchestra.run.smithi131.stdout:The following packages will be upgraded: 2023-09-12T14:40:50.763 INFO:teuthology.orchestra.run.smithi131.stdout: linux-generic linux-headers-generic linux-image-generic 2023-09-12T14:40:50.830 INFO:teuthology.orchestra.run.smithi131.stdout:3 upgraded, 5 newly installed, 0 to remove and 253 not upgraded. 2023-09-12T14:40:50.830 INFO:teuthology.orchestra.run.smithi131.stdout:Need to get 113 MB of archives. 2023-09-12T14:40:50.831 INFO:teuthology.orchestra.run.smithi131.stdout:After this operation, 582 MB of additional disk space will be used. 2023-09-12T14:40:50.831 INFO:teuthology.orchestra.run.smithi131.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-83-generic amd64 5.15.0-83.92 [22.5 MB] 2023-09-12T14:40:51.049 INFO:teuthology.orchestra.run.smithi187.stdout:Reading package lists... 2023-09-12T14:40:51.067 DEBUG:teuthology.orchestra.run.smithi187:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-09-12T14:40:51.128 INFO:teuthology.orchestra.run.smithi187.stdout:Reading package lists... 2023-09-12T14:40:51.257 INFO:teuthology.orchestra.run.smithi131.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-83-generic amd64 5.15.0-83.92 [11.5 MB] 2023-09-12T14:40:51.311 INFO:teuthology.orchestra.run.smithi187.stdout:Building dependency tree... 2023-09-12T14:40:51.311 INFO:teuthology.orchestra.run.smithi187.stdout:Reading state information... 2023-09-12T14:40:51.339 INFO:teuthology.orchestra.run.smithi131.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-83-generic amd64 5.15.0-83.92 [63.9 MB] 2023-09-12T14:40:51.467 INFO:teuthology.orchestra.run.smithi187.stdout:The following packages were automatically installed and are no longer required: 2023-09-12T14:40:51.467 INFO:teuthology.orchestra.run.smithi187.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-09-12T14:40:51.468 INFO:teuthology.orchestra.run.smithi187.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-09-12T14:40:51.468 INFO:teuthology.orchestra.run.smithi187.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-09-12T14:40:51.469 INFO:teuthology.orchestra.run.smithi187.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2023-09-12T14:40:51.469 INFO:teuthology.orchestra.run.smithi187.stdout: python2.7-minimal 2023-09-12T14:40:51.469 INFO:teuthology.orchestra.run.smithi187.stdout:Use 'sudo apt autoremove' to remove them. 2023-09-12T14:40:51.470 INFO:teuthology.orchestra.run.smithi187.stdout:The following additional packages will be installed: 2023-09-12T14:40:51.470 INFO:teuthology.orchestra.run.smithi187.stdout: linux-generic linux-headers-5.15.0-83 linux-headers-5.15.0-83-generic 2023-09-12T14:40:51.470 INFO:teuthology.orchestra.run.smithi187.stdout: linux-headers-generic linux-image-5.15.0-83-generic 2023-09-12T14:40:51.471 INFO:teuthology.orchestra.run.smithi187.stdout: linux-modules-5.15.0-83-generic linux-modules-extra-5.15.0-83-generic 2023-09-12T14:40:51.471 INFO:teuthology.orchestra.run.smithi187.stdout:Suggested packages: 2023-09-12T14:40:51.471 INFO:teuthology.orchestra.run.smithi187.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2023-09-12T14:40:51.471 INFO:teuthology.orchestra.run.smithi187.stdout:Recommended packages: 2023-09-12T14:40:51.472 INFO:teuthology.orchestra.run.smithi187.stdout: thermald 2023-09-12T14:40:51.519 INFO:teuthology.orchestra.run.smithi187.stdout:The following NEW packages will be installed: 2023-09-12T14:40:51.519 INFO:teuthology.orchestra.run.smithi187.stdout: linux-headers-5.15.0-83 linux-headers-5.15.0-83-generic 2023-09-12T14:40:51.519 INFO:teuthology.orchestra.run.smithi187.stdout: linux-image-5.15.0-83-generic linux-modules-5.15.0-83-generic 2023-09-12T14:40:51.520 INFO:teuthology.orchestra.run.smithi187.stdout: linux-modules-extra-5.15.0-83-generic 2023-09-12T14:40:51.520 INFO:teuthology.orchestra.run.smithi187.stdout:The following packages will be upgraded: 2023-09-12T14:40:51.521 INFO:teuthology.orchestra.run.smithi187.stdout: linux-generic linux-headers-generic linux-image-generic 2023-09-12T14:40:51.571 INFO:teuthology.orchestra.run.smithi187.stdout:3 upgraded, 5 newly installed, 0 to remove and 253 not upgraded. 2023-09-12T14:40:51.572 INFO:teuthology.orchestra.run.smithi187.stdout:Need to get 113 MB of archives. 2023-09-12T14:40:51.572 INFO:teuthology.orchestra.run.smithi187.stdout:After this operation, 582 MB of additional disk space will be used. 2023-09-12T14:40:51.572 INFO:teuthology.orchestra.run.smithi187.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-83-generic amd64 5.15.0-83.92 [22.5 MB] 2023-09-12T14:40:52.025 INFO:teuthology.orchestra.run.smithi187.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-83-generic amd64 5.15.0-83.92 [11.5 MB] 2023-09-12T14:40:52.115 INFO:teuthology.orchestra.run.smithi131.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.83.80 [1,698 B] 2023-09-12T14:40:52.115 INFO:teuthology.orchestra.run.smithi131.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.83.80 [2,480 B] 2023-09-12T14:40:52.115 INFO:teuthology.orchestra.run.smithi131.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-83 all 5.15.0-83.92 [12.3 MB] 2023-09-12T14:40:52.125 INFO:teuthology.orchestra.run.smithi187.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-83-generic amd64 5.15.0-83.92 [63.9 MB] 2023-09-12T14:40:52.232 INFO:teuthology.orchestra.run.smithi131.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-83-generic amd64 5.15.0-83.92 [2,852 kB] 2023-09-12T14:40:52.252 INFO:teuthology.orchestra.run.smithi131.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.83.80 [2,320 B] 2023-09-12T14:40:52.629 INFO:teuthology.orchestra.run.smithi131.stdout:Fetched 113 MB in 1s (76.2 MB/s) 2023-09-12T14:40:52.816 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package linux-modules-5.15.0-83-generic. 2023-09-12T14:40:52.917 INFO:teuthology.orchestra.run.smithi187.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.83.80 [1,698 B] 2023-09-12T14:40:52.917 INFO:teuthology.orchestra.run.smithi187.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.83.80 [2,480 B] 2023-09-12T14:40:52.917 INFO:teuthology.orchestra.run.smithi187.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-83 all 5.15.0-83.92 [12.3 MB] 2023-09-12T14:40:53.038 INFO:teuthology.orchestra.run.smithi187.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-83-generic amd64 5.15.0-83.92 [2,852 kB] 2023-09-12T14:40:53.137 INFO:teuthology.orchestra.run.smithi187.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.83.80 [2,320 B] 2023-09-12T14:40:53.564 INFO:teuthology.orchestra.run.smithi187.stdout:Fetched 113 MB in 2s (70.2 MB/s) 2023-09-12T14:40:53.746 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package linux-modules-5.15.0-83-generic. 2023-09-12T14:40:54.750 INFO:teuthology.orchestra.run.smithi131.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2023-09-12T14:40:54.754 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../0-linux-modules-5.15.0-83-generic_5.15.0-83.92_amd64.deb ... 2023-09-12T14:40:54.839 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking linux-modules-5.15.0-83-generic (5.15.0-83.92) ... 2023-09-12T14:40:55.802 INFO:teuthology.orchestra.run.smithi187.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2023-09-12T14:40:55.806 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../0-linux-modules-5.15.0-83-generic_5.15.0-83.92_amd64.deb ... 2023-09-12T14:40:55.943 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking linux-modules-5.15.0-83-generic (5.15.0-83.92) ... 2023-09-12T14:40:57.374 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package linux-image-5.15.0-83-generic. 2023-09-12T14:40:57.396 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../1-linux-image-5.15.0-83-generic_5.15.0-83.92_amd64.deb ... 2023-09-12T14:40:57.481 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking linux-image-5.15.0-83-generic (5.15.0-83.92) ... 2023-09-12T14:40:57.960 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-83-generic. 2023-09-12T14:40:57.980 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-83-generic_5.15.0-83.92_amd64.deb ... 2023-09-12T14:40:58.018 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking linux-modules-extra-5.15.0-83-generic (5.15.0-83.92) ... 2023-09-12T14:40:58.495 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package linux-image-5.15.0-83-generic. 2023-09-12T14:40:58.516 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../1-linux-image-5.15.0-83-generic_5.15.0-83.92_amd64.deb ... 2023-09-12T14:40:58.602 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking linux-image-5.15.0-83-generic (5.15.0-83.92) ... 2023-09-12T14:40:59.125 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-83-generic. 2023-09-12T14:40:59.145 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-83-generic_5.15.0-83.92_amd64.deb ... 2023-09-12T14:40:59.183 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking linux-modules-extra-5.15.0-83-generic (5.15.0-83.92) ... 2023-09-12T14:41:04.373 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../3-linux-generic_5.15.0.83.80_amd64.deb ... 2023-09-12T14:41:04.489 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking linux-generic (5.15.0.83.80) over (5.15.0.56.54) ... 2023-09-12T14:41:04.815 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.83.80_amd64.deb ... 2023-09-12T14:41:04.943 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking linux-image-generic (5.15.0.83.80) over (5.15.0.56.54) ... 2023-09-12T14:41:05.246 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package linux-headers-5.15.0-83. 2023-09-12T14:41:05.269 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../5-linux-headers-5.15.0-83_5.15.0-83.92_all.deb ... 2023-09-12T14:41:05.313 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking linux-headers-5.15.0-83 (5.15.0-83.92) ... 2023-09-12T14:41:05.913 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../3-linux-generic_5.15.0.83.80_amd64.deb ... 2023-09-12T14:41:06.054 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking linux-generic (5.15.0.83.80) over (5.15.0.56.54) ... 2023-09-12T14:41:06.384 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.83.80_amd64.deb ... 2023-09-12T14:41:06.517 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking linux-image-generic (5.15.0.83.80) over (5.15.0.56.54) ... 2023-09-12T14:41:06.811 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package linux-headers-5.15.0-83. 2023-09-12T14:41:06.833 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../5-linux-headers-5.15.0-83_5.15.0-83.92_all.deb ... 2023-09-12T14:41:06.878 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking linux-headers-5.15.0-83 (5.15.0-83.92) ... 2023-09-12T14:41:10.964 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package linux-headers-5.15.0-83-generic. 2023-09-12T14:41:11.001 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../6-linux-headers-5.15.0-83-generic_5.15.0-83.92_amd64.deb ... 2023-09-12T14:41:11.039 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking linux-headers-5.15.0-83-generic (5.15.0-83.92) ... 2023-09-12T14:41:13.498 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package linux-headers-5.15.0-83-generic. 2023-09-12T14:41:13.534 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../6-linux-headers-5.15.0-83-generic_5.15.0-83.92_amd64.deb ... 2023-09-12T14:41:13.581 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking linux-headers-5.15.0-83-generic (5.15.0-83.92) ... 2023-09-12T14:41:13.760 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.83.80_amd64.deb ... 2023-09-12T14:41:13.885 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking linux-headers-generic (5.15.0.83.80) over (5.15.0.56.54) ... 2023-09-12T14:41:14.201 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up linux-headers-5.15.0-83 (5.15.0-83.92) ... 2023-09-12T14:41:14.332 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up linux-headers-5.15.0-83-generic (5.15.0-83.92) ... 2023-09-12T14:41:14.458 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up linux-headers-generic (5.15.0.83.80) ... 2023-09-12T14:41:14.587 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up linux-image-5.15.0-83-generic (5.15.0-83.92) ... 2023-09-12T14:41:15.894 INFO:teuthology.orchestra.run.smithi131.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2023-09-12T14:41:15.895 INFO:teuthology.orchestra.run.smithi131.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2023-09-12T14:41:15.895 INFO:teuthology.orchestra.run.smithi131.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-83-generic 2023-09-12T14:41:15.895 INFO:teuthology.orchestra.run.smithi131.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-83-generic 2023-09-12T14:41:16.055 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up linux-modules-5.15.0-83-generic (5.15.0-83.92) ... 2023-09-12T14:41:16.516 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.83.80_amd64.deb ... 2023-09-12T14:41:16.650 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking linux-headers-generic (5.15.0.83.80) over (5.15.0.56.54) ... 2023-09-12T14:41:16.999 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up linux-headers-5.15.0-83 (5.15.0-83.92) ... 2023-09-12T14:41:17.121 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up linux-headers-5.15.0-83-generic (5.15.0-83.92) ... 2023-09-12T14:41:17.272 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up linux-headers-generic (5.15.0.83.80) ... 2023-09-12T14:41:17.426 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up linux-image-5.15.0-83-generic (5.15.0-83.92) ... 2023-09-12T14:41:17.441 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up linux-modules-extra-5.15.0-83-generic (5.15.0-83.92) ... 2023-09-12T14:41:18.839 INFO:teuthology.orchestra.run.smithi187.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2023-09-12T14:41:18.839 INFO:teuthology.orchestra.run.smithi187.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2023-09-12T14:41:18.839 INFO:teuthology.orchestra.run.smithi187.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-83-generic 2023-09-12T14:41:18.839 INFO:teuthology.orchestra.run.smithi187.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-83-generic 2023-09-12T14:41:19.033 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up linux-image-generic (5.15.0.83.80) ... 2023-09-12T14:41:19.061 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up linux-modules-5.15.0-83-generic (5.15.0-83.92) ... 2023-09-12T14:41:19.159 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up linux-generic (5.15.0.83.80) ... 2023-09-12T14:41:19.285 INFO:teuthology.orchestra.run.smithi131.stdout:Processing triggers for linux-image-5.15.0-83-generic (5.15.0-83.92) ... 2023-09-12T14:41:19.350 INFO:teuthology.orchestra.run.smithi131.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-09-12T14:41:19.350 INFO:teuthology.orchestra.run.smithi131.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-83-generic 2023-09-12T14:41:20.438 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up linux-modules-extra-5.15.0-83-generic (5.15.0-83.92) ... 2023-09-12T14:41:22.300 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up linux-image-generic (5.15.0.83.80) ... 2023-09-12T14:41:22.430 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up linux-generic (5.15.0.83.80) ... 2023-09-12T14:41:22.590 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for linux-image-5.15.0-83-generic (5.15.0-83.92) ... 2023-09-12T14:41:22.662 INFO:teuthology.orchestra.run.smithi187.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-09-12T14:41:22.662 INFO:teuthology.orchestra.run.smithi187.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-83-generic 2023-09-12T14:41:34.367 INFO:teuthology.orchestra.run.smithi131.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-09-12T14:41:34.368 INFO:teuthology.orchestra.run.smithi131.stdout:Sourcing file `/etc/default/grub' 2023-09-12T14:41:34.388 INFO:teuthology.orchestra.run.smithi131.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-09-12T14:41:34.403 INFO:teuthology.orchestra.run.smithi131.stdout:Generating grub configuration file ... 2023-09-12T14:41:34.704 INFO:teuthology.orchestra.run.smithi131.stdout:Found linux image: /boot/vmlinuz-5.15.0-83-generic 2023-09-12T14:41:34.723 INFO:teuthology.orchestra.run.smithi131.stdout:Found initrd image: /boot/initrd.img-5.15.0-83-generic 2023-09-12T14:41:35.018 INFO:teuthology.orchestra.run.smithi131.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-09-12T14:41:35.024 INFO:teuthology.orchestra.run.smithi131.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-09-12T14:41:35.095 INFO:teuthology.orchestra.run.smithi131.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-09-12T14:41:35.100 INFO:teuthology.orchestra.run.smithi131.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-09-12T14:41:35.286 INFO:teuthology.orchestra.run.smithi131.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2023-09-12T14:41:35.287 INFO:teuthology.orchestra.run.smithi131.stdout:Systems on them will not be added to the GRUB boot configuration. 2023-09-12T14:41:35.287 INFO:teuthology.orchestra.run.smithi131.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-09-12T14:41:35.329 INFO:teuthology.orchestra.run.smithi131.stdout:done 2023-09-12T14:41:35.778 DEBUG:teuthology.orchestra.run.smithi131:> dpkg -s linux-image-generic 2023-09-12T14:41:35.802 INFO:teuthology.orchestra.run.smithi131.stdout:Package: linux-image-generic 2023-09-12T14:41:35.802 INFO:teuthology.orchestra.run.smithi131.stdout:Status: install ok installed 2023-09-12T14:41:35.803 INFO:teuthology.orchestra.run.smithi131.stdout:Priority: optional 2023-09-12T14:41:35.803 INFO:teuthology.orchestra.run.smithi131.stdout:Section: kernel 2023-09-12T14:41:35.803 INFO:teuthology.orchestra.run.smithi131.stdout:Installed-Size: 20 2023-09-12T14:41:35.803 INFO:teuthology.orchestra.run.smithi131.stdout:Maintainer: Ubuntu Kernel Team 2023-09-12T14:41:35.803 INFO:teuthology.orchestra.run.smithi131.stdout:Architecture: amd64 2023-09-12T14:41:35.804 INFO:teuthology.orchestra.run.smithi131.stdout:Source: linux-meta 2023-09-12T14:41:35.804 INFO:teuthology.orchestra.run.smithi131.stdout:Version: 5.15.0.83.80 2023-09-12T14:41:35.804 INFO:teuthology.orchestra.run.smithi131.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-83), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-09-12T14:41:35.804 INFO:teuthology.orchestra.run.smithi131.stdout:Depends: linux-image-5.15.0-83-generic, linux-modules-extra-5.15.0-83-generic, linux-firmware, intel-microcode, amd64-microcode 2023-09-12T14:41:35.804 INFO:teuthology.orchestra.run.smithi131.stdout:Recommends: thermald 2023-09-12T14:41:35.805 INFO:teuthology.orchestra.run.smithi131.stdout:Description: Generic Linux kernel image 2023-09-12T14:41:35.805 INFO:teuthology.orchestra.run.smithi131.stdout: This package will always depend on the latest generic kernel image 2023-09-12T14:41:35.805 INFO:teuthology.orchestra.run.smithi131.stdout: available. 2023-09-12T14:41:35.805 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-83-generic 2023-09-12T14:41:35.806 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-09-12T14:41:35.806 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-09-12T14:41:35.806 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-09-12T14:41:35.806 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi131.front.sepia.ceph.com, path=None, version=distro) 2023-09-12T14:41:35.807 DEBUG:teuthology.orchestra.run.smithi131:> sudo apt-get clean 2023-09-12T14:41:36.006 DEBUG:teuthology.orchestra.run.smithi131:> sudo apt-get update 2023-09-12T14:41:36.135 INFO:teuthology.orchestra.run.smithi131.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-09-12T14:41:36.146 INFO:teuthology.orchestra.run.smithi131.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-09-12T14:41:36.147 INFO:teuthology.orchestra.run.smithi131.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-09-12T14:41:36.163 INFO:teuthology.orchestra.run.smithi131.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-09-12T14:41:36.913 INFO:teuthology.orchestra.run.smithi187.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-09-12T14:41:36.913 INFO:teuthology.orchestra.run.smithi187.stdout:Sourcing file `/etc/default/grub' 2023-09-12T14:41:36.938 INFO:teuthology.orchestra.run.smithi187.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-09-12T14:41:36.959 INFO:teuthology.orchestra.run.smithi187.stdout:Generating grub configuration file ... 2023-09-12T14:41:37.339 INFO:teuthology.orchestra.run.smithi187.stdout:Found linux image: /boot/vmlinuz-5.15.0-83-generic 2023-09-12T14:41:37.362 INFO:teuthology.orchestra.run.smithi187.stdout:Found initrd image: /boot/initrd.img-5.15.0-83-generic 2023-09-12T14:41:37.397 INFO:teuthology.orchestra.run.smithi131.stdout:Reading package lists... 2023-09-12T14:41:37.415 DEBUG:teuthology.orchestra.run.smithi131:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-09-12T14:41:37.474 INFO:teuthology.orchestra.run.smithi131.stdout:Reading package lists... 2023-09-12T14:41:37.655 INFO:teuthology.orchestra.run.smithi131.stdout:Building dependency tree... 2023-09-12T14:41:37.655 INFO:teuthology.orchestra.run.smithi131.stdout:Reading state information... 2023-09-12T14:41:37.812 INFO:teuthology.orchestra.run.smithi187.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-09-12T14:41:37.814 INFO:teuthology.orchestra.run.smithi131.stdout:linux-image-generic is already the newest version (5.15.0.83.80). 2023-09-12T14:41:37.814 INFO:teuthology.orchestra.run.smithi131.stdout:The following packages were automatically installed and are no longer required: 2023-09-12T14:41:37.814 INFO:teuthology.orchestra.run.smithi131.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-09-12T14:41:37.814 INFO:teuthology.orchestra.run.smithi131.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-09-12T14:41:37.815 INFO:teuthology.orchestra.run.smithi131.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-09-12T14:41:37.815 INFO:teuthology.orchestra.run.smithi131.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-09-12T14:41:37.815 INFO:teuthology.orchestra.run.smithi131.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-09-12T14:41:37.816 INFO:teuthology.orchestra.run.smithi131.stdout:Use 'sudo apt autoremove' to remove them. 2023-09-12T14:41:37.818 INFO:teuthology.orchestra.run.smithi187.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-09-12T14:41:37.854 INFO:teuthology.orchestra.run.smithi131.stdout:0 upgraded, 0 newly installed, 0 to remove and 253 not upgraded. 2023-09-12T14:41:37.855 DEBUG:teuthology.orchestra.run.smithi131:> dpkg -s linux-image-generic 2023-09-12T14:41:37.893 INFO:teuthology.orchestra.run.smithi187.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-09-12T14:41:37.898 INFO:teuthology.orchestra.run.smithi187.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-09-12T14:41:37.912 INFO:teuthology.orchestra.run.smithi131.stdout:Package: linux-image-generic 2023-09-12T14:41:37.913 INFO:teuthology.orchestra.run.smithi131.stdout:Status: install ok installed 2023-09-12T14:41:37.913 INFO:teuthology.orchestra.run.smithi131.stdout:Priority: optional 2023-09-12T14:41:37.913 INFO:teuthology.orchestra.run.smithi131.stdout:Section: kernel 2023-09-12T14:41:37.913 INFO:teuthology.orchestra.run.smithi131.stdout:Installed-Size: 20 2023-09-12T14:41:37.913 INFO:teuthology.orchestra.run.smithi131.stdout:Maintainer: Ubuntu Kernel Team 2023-09-12T14:41:37.914 INFO:teuthology.orchestra.run.smithi131.stdout:Architecture: amd64 2023-09-12T14:41:37.914 INFO:teuthology.orchestra.run.smithi131.stdout:Source: linux-meta 2023-09-12T14:41:37.914 INFO:teuthology.orchestra.run.smithi131.stdout:Version: 5.15.0.83.80 2023-09-12T14:41:37.914 INFO:teuthology.orchestra.run.smithi131.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-83), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-09-12T14:41:37.915 INFO:teuthology.orchestra.run.smithi131.stdout:Depends: linux-image-5.15.0-83-generic, linux-modules-extra-5.15.0-83-generic, linux-firmware, intel-microcode, amd64-microcode 2023-09-12T14:41:37.915 INFO:teuthology.orchestra.run.smithi131.stdout:Recommends: thermald 2023-09-12T14:41:37.915 INFO:teuthology.orchestra.run.smithi131.stdout:Description: Generic Linux kernel image 2023-09-12T14:41:37.915 INFO:teuthology.orchestra.run.smithi131.stdout: This package will always depend on the latest generic kernel image 2023-09-12T14:41:37.915 INFO:teuthology.orchestra.run.smithi131.stdout: available. 2023-09-12T14:41:37.916 DEBUG:teuthology.orchestra.run.smithi131:> mktemp 2023-09-12T14:41:37.961 INFO:teuthology.orchestra.run.smithi131.stdout:/tmp/tmp.dYIQNdijGD 2023-09-12T14:41:37.961 DEBUG:teuthology.orchestra.run.smithi131:> sudo cp /boot/grub/grub.cfg /tmp/tmp.dYIQNdijGD 2023-09-12T14:41:38.017 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0666 /tmp/tmp.dYIQNdijGD 2023-09-12T14:41:38.074 INFO:teuthology.orchestra.run.smithi187.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2023-09-12T14:41:38.074 INFO:teuthology.orchestra.run.smithi187.stdout:Systems on them will not be added to the GRUB boot configuration. 2023-09-12T14:41:38.074 INFO:teuthology.orchestra.run.smithi187.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-09-12T14:41:38.115 INFO:teuthology.orchestra.run.smithi187.stdout:done 2023-09-12T14:41:38.148 DEBUG:teuthology.orchestra.remote:smithi131:/tmp/tmp.dYIQNdijGD is 10KB 2023-09-12T14:41:38.161 DEBUG:teuthology.orchestra.run.smithi131:> rm -fr /tmp/tmp.dYIQNdijGD 2023-09-12T14:41:38.167 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-09-12T14:41:38.221 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:41:38.221 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-09-12T14:41:38.222 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-09-12T14:41:38.288 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-83-generic 2023-09-12T14:41:38.288 DEBUG:teuthology.orchestra.run.smithi131:> sudo update-grub 2023-09-12T14:41:38.629 DEBUG:teuthology.orchestra.run.smithi187:> dpkg -s linux-image-generic 2023-09-12T14:41:38.653 INFO:teuthology.orchestra.run.smithi187.stdout:Package: linux-image-generic 2023-09-12T14:41:38.653 INFO:teuthology.orchestra.run.smithi187.stdout:Status: install ok installed 2023-09-12T14:41:38.653 INFO:teuthology.orchestra.run.smithi187.stdout:Priority: optional 2023-09-12T14:41:38.653 INFO:teuthology.orchestra.run.smithi187.stdout:Section: kernel 2023-09-12T14:41:38.653 INFO:teuthology.orchestra.run.smithi187.stdout:Installed-Size: 20 2023-09-12T14:41:38.654 INFO:teuthology.orchestra.run.smithi187.stdout:Maintainer: Ubuntu Kernel Team 2023-09-12T14:41:38.654 INFO:teuthology.orchestra.run.smithi187.stdout:Architecture: amd64 2023-09-12T14:41:38.654 INFO:teuthology.orchestra.run.smithi187.stdout:Source: linux-meta 2023-09-12T14:41:38.654 INFO:teuthology.orchestra.run.smithi187.stdout:Version: 5.15.0.83.80 2023-09-12T14:41:38.655 INFO:teuthology.orchestra.run.smithi187.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-83), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-09-12T14:41:38.655 INFO:teuthology.orchestra.run.smithi187.stdout:Depends: linux-image-5.15.0-83-generic, linux-modules-extra-5.15.0-83-generic, linux-firmware, intel-microcode, amd64-microcode 2023-09-12T14:41:38.655 INFO:teuthology.orchestra.run.smithi187.stdout:Recommends: thermald 2023-09-12T14:41:38.655 INFO:teuthology.orchestra.run.smithi187.stdout:Description: Generic Linux kernel image 2023-09-12T14:41:38.655 INFO:teuthology.orchestra.run.smithi187.stdout: This package will always depend on the latest generic kernel image 2023-09-12T14:41:38.656 INFO:teuthology.orchestra.run.smithi187.stdout: available. 2023-09-12T14:41:38.656 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-83-generic 2023-09-12T14:41:38.656 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-09-12T14:41:38.656 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-09-12T14:41:38.656 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-09-12T14:41:38.657 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi187.front.sepia.ceph.com, path=None, version=distro) 2023-09-12T14:41:38.657 DEBUG:teuthology.orchestra.run.smithi187:> sudo apt-get clean 2023-09-12T14:41:38.852 DEBUG:teuthology.orchestra.run.smithi187:> sudo apt-get update 2023-09-12T14:41:38.971 INFO:teuthology.orchestra.run.smithi187.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-09-12T14:41:38.979 INFO:teuthology.orchestra.run.smithi187.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-09-12T14:41:38.996 INFO:teuthology.orchestra.run.smithi187.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-09-12T14:41:39.093 INFO:teuthology.orchestra.run.smithi187.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-09-12T14:41:39.461 INFO:teuthology.orchestra.run.smithi131.stderr:Sourcing file `/etc/default/grub' 2023-09-12T14:41:39.462 INFO:teuthology.orchestra.run.smithi131.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-09-12T14:41:39.464 INFO:teuthology.orchestra.run.smithi131.stderr:Generating grub configuration file ... 2023-09-12T14:41:39.624 INFO:teuthology.orchestra.run.smithi131.stderr:Found linux image: /boot/vmlinuz-5.15.0-83-generic 2023-09-12T14:41:39.629 INFO:teuthology.orchestra.run.smithi131.stderr:Found initrd image: /boot/initrd.img-5.15.0-83-generic 2023-09-12T14:41:39.840 INFO:teuthology.orchestra.run.smithi131.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-09-12T14:41:39.846 INFO:teuthology.orchestra.run.smithi131.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-09-12T14:41:39.885 INFO:teuthology.orchestra.run.smithi131.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-09-12T14:41:39.890 INFO:teuthology.orchestra.run.smithi131.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-09-12T14:41:40.004 INFO:teuthology.orchestra.run.smithi131.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-09-12T14:41:40.004 INFO:teuthology.orchestra.run.smithi131.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-09-12T14:41:40.005 INFO:teuthology.orchestra.run.smithi131.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-09-12T14:41:40.021 INFO:teuthology.orchestra.run.smithi131.stderr:done 2023-09-12T14:41:40.039 DEBUG:teuthology.orchestra.run.smithi131:> sudo shutdown -r now 2023-09-12T14:41:40.226 INFO:teuthology.orchestra.run.smithi187.stdout:Reading package lists... 2023-09-12T14:41:40.244 DEBUG:teuthology.orchestra.run.smithi187:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-09-12T14:41:40.294 INFO:teuthology.orchestra.run.smithi187.stdout:Reading package lists... 2023-09-12T14:41:40.456 INFO:teuthology.orchestra.run.smithi187.stdout:Building dependency tree... 2023-09-12T14:41:40.457 INFO:teuthology.orchestra.run.smithi187.stdout:Reading state information... 2023-09-12T14:41:40.596 INFO:teuthology.orchestra.run.smithi187.stdout:linux-image-generic is already the newest version (5.15.0.83.80). 2023-09-12T14:41:40.596 INFO:teuthology.orchestra.run.smithi187.stdout:The following packages were automatically installed and are no longer required: 2023-09-12T14:41:40.597 INFO:teuthology.orchestra.run.smithi187.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-09-12T14:41:40.597 INFO:teuthology.orchestra.run.smithi187.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-09-12T14:41:40.597 INFO:teuthology.orchestra.run.smithi187.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-09-12T14:41:40.597 INFO:teuthology.orchestra.run.smithi187.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-09-12T14:41:40.597 INFO:teuthology.orchestra.run.smithi187.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-09-12T14:41:40.598 INFO:teuthology.orchestra.run.smithi187.stdout:Use 'sudo apt autoremove' to remove them. 2023-09-12T14:41:40.631 INFO:teuthology.orchestra.run.smithi187.stdout:0 upgraded, 0 newly installed, 0 to remove and 253 not upgraded. 2023-09-12T14:41:40.633 DEBUG:teuthology.orchestra.run.smithi187:> dpkg -s linux-image-generic 2023-09-12T14:41:40.691 INFO:teuthology.orchestra.run.smithi187.stdout:Package: linux-image-generic 2023-09-12T14:41:40.692 INFO:teuthology.orchestra.run.smithi187.stdout:Status: install ok installed 2023-09-12T14:41:40.692 INFO:teuthology.orchestra.run.smithi187.stdout:Priority: optional 2023-09-12T14:41:40.692 INFO:teuthology.orchestra.run.smithi187.stdout:Section: kernel 2023-09-12T14:41:40.692 INFO:teuthology.orchestra.run.smithi187.stdout:Installed-Size: 20 2023-09-12T14:41:40.693 INFO:teuthology.orchestra.run.smithi187.stdout:Maintainer: Ubuntu Kernel Team 2023-09-12T14:41:40.693 INFO:teuthology.orchestra.run.smithi187.stdout:Architecture: amd64 2023-09-12T14:41:40.693 INFO:teuthology.orchestra.run.smithi187.stdout:Source: linux-meta 2023-09-12T14:41:40.693 INFO:teuthology.orchestra.run.smithi187.stdout:Version: 5.15.0.83.80 2023-09-12T14:41:40.693 INFO:teuthology.orchestra.run.smithi187.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-83), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-09-12T14:41:40.694 INFO:teuthology.orchestra.run.smithi187.stdout:Depends: linux-image-5.15.0-83-generic, linux-modules-extra-5.15.0-83-generic, linux-firmware, intel-microcode, amd64-microcode 2023-09-12T14:41:40.694 INFO:teuthology.orchestra.run.smithi187.stdout:Recommends: thermald 2023-09-12T14:41:40.694 INFO:teuthology.orchestra.run.smithi187.stdout:Description: Generic Linux kernel image 2023-09-12T14:41:40.694 INFO:teuthology.orchestra.run.smithi187.stdout: This package will always depend on the latest generic kernel image 2023-09-12T14:41:40.694 INFO:teuthology.orchestra.run.smithi187.stdout: available. 2023-09-12T14:41:40.695 DEBUG:teuthology.orchestra.run.smithi187:> mktemp 2023-09-12T14:41:40.737 INFO:teuthology.orchestra.run.smithi187.stdout:/tmp/tmp.7g3WjyARLG 2023-09-12T14:41:40.738 DEBUG:teuthology.orchestra.run.smithi187:> sudo cp /boot/grub/grub.cfg /tmp/tmp.7g3WjyARLG 2023-09-12T14:41:40.794 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0666 /tmp/tmp.7g3WjyARLG 2023-09-12T14:41:40.902 DEBUG:teuthology.orchestra.remote:smithi187:/tmp/tmp.7g3WjyARLG is 10KB 2023-09-12T14:41:40.915 DEBUG:teuthology.orchestra.run.smithi187:> rm -fr /tmp/tmp.7g3WjyARLG 2023-09-12T14:41:40.920 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-09-12T14:41:40.972 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:41:40.972 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-09-12T14:41:40.973 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-09-12T14:41:41.057 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-83-generic 2023-09-12T14:41:41.058 DEBUG:teuthology.orchestra.run.smithi187:> sudo update-grub 2023-09-12T14:41:42.393 INFO:teuthology.orchestra.run.smithi187.stderr:Sourcing file `/etc/default/grub' 2023-09-12T14:41:42.403 INFO:teuthology.orchestra.run.smithi187.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-09-12T14:41:42.412 INFO:teuthology.orchestra.run.smithi187.stderr:Generating grub configuration file ... 2023-09-12T14:41:42.621 INFO:teuthology.orchestra.run.smithi187.stderr:Found linux image: /boot/vmlinuz-5.15.0-83-generic 2023-09-12T14:41:42.627 INFO:teuthology.orchestra.run.smithi187.stderr:Found initrd image: /boot/initrd.img-5.15.0-83-generic 2023-09-12T14:41:42.978 INFO:teuthology.orchestra.run.smithi187.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-09-12T14:41:42.983 INFO:teuthology.orchestra.run.smithi187.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-09-12T14:41:43.025 INFO:teuthology.orchestra.run.smithi187.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-09-12T14:41:43.031 INFO:teuthology.orchestra.run.smithi187.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-09-12T14:41:43.151 INFO:teuthology.orchestra.run.smithi187.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-09-12T14:41:43.151 INFO:teuthology.orchestra.run.smithi187.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-09-12T14:41:43.151 INFO:teuthology.orchestra.run.smithi187.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-09-12T14:41:43.162 INFO:teuthology.orchestra.run.smithi187.stderr:done 2023-09-12T14:41:43.170 DEBUG:teuthology.orchestra.run.smithi187:> sudo shutdown -r now 2023-09-12T14:42:10.057 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-09-12T14:42:10.058 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2023-09-12T14:42:10.058 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-12T14:42:13.218 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-09-12T14:42:13.219 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2023-09-12T14:42:13.219 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-12T14:42:28.382 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.131 2023-09-12T14:42:31.706 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.187 2023-09-12T14:42:37.389 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2023-09-12T14:42:37.390 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-12T14:42:40.442 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.131 2023-09-12T14:42:40.708 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2023-09-12T14:42:40.708 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-12T14:42:43.770 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.187 2023-09-12T14:42:52.451 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2023-09-12T14:42:52.452 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-12T14:42:55.774 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2023-09-12T14:42:55.775 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-12T14:43:14.202 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.187 2023-09-12T14:43:24.167 DEBUG:teuthology.orchestra.run.smithi131:> true 2023-09-12T14:43:24.785 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2023-09-12T14:43:24.785 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-83-generic"... 2023-09-12T14:43:24.785 DEBUG:teuthology.orchestra.run.smithi131:> uname -r 2023-09-12T14:43:24.830 INFO:teuthology.orchestra.run.smithi131.stdout:5.15.0-83-generic 2023-09-12T14:43:24.830 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-83-generic vs 5.15.0-83-generic 2023-09-12T14:43:24.831 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-09-12T14:43:24.831 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-09-12T14:43:25.832 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-09-12T14:43:25.832 DEBUG:teuthology.orchestra.run.smithi131:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-09-12T14:43:26.032 INFO:teuthology.orchestra.run.smithi131.stdout:ttyS1 2023-09-12T14:43:26.071 DEBUG:teuthology.parallel:result is None 2023-09-12T14:43:29.203 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2023-09-12T14:43:29.204 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-12T14:43:30.193 DEBUG:teuthology.orchestra.run.smithi187:> true 2023-09-12T14:43:31.298 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi187.front.sepia.ceph.com' 2023-09-12T14:43:31.298 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-83-generic"... 2023-09-12T14:43:31.298 DEBUG:teuthology.orchestra.run.smithi187:> uname -r 2023-09-12T14:43:31.344 INFO:teuthology.orchestra.run.smithi187.stdout:5.15.0-83-generic 2023-09-12T14:43:31.344 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-83-generic vs 5.15.0-83-generic 2023-09-12T14:43:31.345 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-09-12T14:43:31.345 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-09-12T14:43:32.345 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-09-12T14:43:32.346 DEBUG:teuthology.orchestra.run.smithi187:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-09-12T14:43:32.431 INFO:teuthology.orchestra.run.smithi187.stdout:ttyS1 2023-09-12T14:43:32.465 DEBUG:teuthology.parallel:result is None 2023-09-12T14:43:32.466 INFO:teuthology.run_tasks:Running task internal.base... 2023-09-12T14:43:32.472 INFO:teuthology.task.internal:Creating test directory... 2023-09-12T14:43:32.473 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-12T14:43:32.476 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-12T14:43:32.481 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-09-12T14:43:32.487 INFO:teuthology.run_tasks:Running task internal.archive... 2023-09-12T14:43:32.499 INFO:teuthology.task.internal:Creating archive directory... 2023-09-12T14:43:32.499 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-12T14:43:32.524 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-12T14:43:32.627 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-09-12T14:43:32.633 INFO:teuthology.task.internal:Enabling coredump saving... 2023-09-12T14:43:32.634 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-12T14:43:32.636 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-12T14:43:32.653 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-12T14:43:32.663 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-12T14:43:32.667 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-12T14:43:32.675 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-12T14:43:32.677 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-09-12T14:43:32.684 INFO:teuthology.task.internal:Configuring sudo... 2023-09-12T14:43:32.684 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-12T14:43:32.707 DEBUG:teuthology.orchestra.run.smithi187:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-12T14:43:32.733 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-09-12T14:43:32.741 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-09-12T14:43:32.741 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-12T14:43:32.763 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-12T14:43:32.780 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-12T14:43:32.814 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-12T14:43:32.862 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:43:32.863 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-12T14:43:32.947 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-12T14:43:32.953 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-12T14:43:33.000 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:43:33.001 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-12T14:43:33.056 DEBUG:teuthology.orchestra.run.smithi131:> sudo service rsyslog restart 2023-09-12T14:43:33.059 DEBUG:teuthology.orchestra.run.smithi187:> sudo service rsyslog restart 2023-09-12T14:43:33.191 INFO:teuthology.run_tasks:Running task internal.timer... 2023-09-12T14:43:33.204 INFO:teuthology.task.internal:Starting timer... 2023-09-12T14:43:33.205 INFO:teuthology.run_tasks:Running task pcp... 2023-09-12T14:43:33.214 INFO:teuthology.run_tasks:Running task selinux... 2023-09-12T14:43:33.223 DEBUG:teuthology.task.selinux:Excluding smithi131: OS 'ubuntu' does not support SELinux 2023-09-12T14:43:33.224 DEBUG:teuthology.task.selinux:Excluding smithi187: OS 'ubuntu' does not support SELinux 2023-09-12T14:43:33.224 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-09-12T14:43:33.225 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-09-12T14:43:33.225 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-09-12T14:43:33.225 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-09-12T14:43:33.234 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-09-12T14:43:33.240 INFO:teuthology.repo_utils:Fetching main from origin 2023-09-12T14:43:33.353 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-09-12T14:43:33.367 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-09-12T14:43:33.368 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi131.front.sepia.ceph.com,smithi187.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-09-12T14:51:12.420 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi131.front.sepia.ceph.com'), Remote(name='ubuntu@smithi187.front.sepia.ceph.com')] 2023-09-12T14:51:12.421 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2023-09-12T14:51:12.422 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-12T14:51:12.501 DEBUG:teuthology.orchestra.run.smithi131:> true 2023-09-12T14:51:12.602 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2023-09-12T14:51:12.603 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2023-09-12T14:51:12.603 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-12T14:51:12.679 DEBUG:teuthology.orchestra.run.smithi187:> true 2023-09-12T14:51:12.790 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi187.front.sepia.ceph.com' 2023-09-12T14:51:12.790 INFO:teuthology.run_tasks:Running task clock... 2023-09-12T14:51:12.800 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-09-12T14:51:12.801 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-12T14:51:12.801 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-12T14:51:12.804 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-12T14:51:12.804 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-12T14:51:12.831 INFO:teuthology.orchestra.run.smithi131.stdout:12 Sep 14:51:12 ntpd[27872]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-09-12T14:51:12.831 INFO:teuthology.orchestra.run.smithi131.stdout:12 Sep 14:51:12 ntpd[27872]: Command line: ntpd -gq 2023-09-12T14:51:12.831 INFO:teuthology.orchestra.run.smithi131.stdout:12 Sep 14:51:12 ntpd[27872]: ---------------------------------------------------- 2023-09-12T14:51:12.831 INFO:teuthology.orchestra.run.smithi131.stdout:12 Sep 14:51:12 ntpd[27872]: ntp-4 is maintained by Network Time Foundation, 2023-09-12T14:51:12.832 INFO:teuthology.orchestra.run.smithi131.stdout:12 Sep 14:51:12 ntpd[27872]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-09-12T14:51:12.832 INFO:teuthology.orchestra.run.smithi131.stdout:12 Sep 14:51:12 ntpd[27872]: corporation. Support and training for ntp-4 are 2023-09-12T14:51:12.832 INFO:teuthology.orchestra.run.smithi131.stdout:12 Sep 14:51:12 ntpd[27872]: available at https://www.nwtime.org/support 2023-09-12T14:51:12.832 INFO:teuthology.orchestra.run.smithi131.stdout:12 Sep 14:51:12 ntpd[27872]: ---------------------------------------------------- 2023-09-12T14:51:12.833 INFO:teuthology.orchestra.run.smithi131.stdout:12 Sep 14:51:12 ntpd[27872]: proto: precision = 0.059 usec (-24) 2023-09-12T14:51:12.833 INFO:teuthology.orchestra.run.smithi131.stdout:12 Sep 14:51:12 ntpd[27872]: basedate set to 2022-02-04 2023-09-12T14:51:12.833 INFO:teuthology.orchestra.run.smithi131.stdout:12 Sep 14:51:12 ntpd[27872]: gps base set to 2022-02-06 (week 2196) 2023-09-12T14:51:12.834 INFO:teuthology.orchestra.run.smithi131.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-09-12T14:51:12.834 INFO:teuthology.orchestra.run.smithi131.stdout:12 Sep 14:51:12 ntpd[27872]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-09-12T14:51:12.835 INFO:teuthology.orchestra.run.smithi131.stderr:restrict ::: KOD does nothing without LIMITED. 2023-09-12T14:51:12.836 INFO:teuthology.orchestra.run.smithi131.stdout:12 Sep 14:51:12 ntpd[27872]: restrict ::: KOD does nothing without LIMITED. 2023-09-12T14:51:12.836 INFO:teuthology.orchestra.run.smithi131.stdout:12 Sep 14:51:12 ntpd[27872]: Listen and drop on 0 v6wildcard [::]:123 2023-09-12T14:51:12.836 INFO:teuthology.orchestra.run.smithi131.stdout:12 Sep 14:51:12 ntpd[27872]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-09-12T14:51:12.836 INFO:teuthology.orchestra.run.smithi131.stdout:12 Sep 14:51:12 ntpd[27872]: Listen normally on 2 lo 127.0.0.1:123 2023-09-12T14:51:12.837 INFO:teuthology.orchestra.run.smithi131.stdout:12 Sep 14:51:12 ntpd[27872]: Listen normally on 3 enp3s0f1 172.21.15.131:123 2023-09-12T14:51:12.837 INFO:teuthology.orchestra.run.smithi131.stdout:12 Sep 14:51:12 ntpd[27872]: Listen normally on 4 lo [::1]:123 2023-09-12T14:51:12.837 INFO:teuthology.orchestra.run.smithi131.stdout:12 Sep 14:51:12 ntpd[27872]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:72f9%5]:123 2023-09-12T14:51:12.837 INFO:teuthology.orchestra.run.smithi131.stdout:12 Sep 14:51:12 ntpd[27872]: Listening on routing socket on fd #22 for interface updates 2023-09-12T14:51:12.863 INFO:teuthology.orchestra.run.smithi187.stdout:12 Sep 14:51:12 ntpd[27821]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-09-12T14:51:12.864 INFO:teuthology.orchestra.run.smithi187.stdout:12 Sep 14:51:12 ntpd[27821]: Command line: ntpd -gq 2023-09-12T14:51:12.864 INFO:teuthology.orchestra.run.smithi187.stdout:12 Sep 14:51:12 ntpd[27821]: ---------------------------------------------------- 2023-09-12T14:51:12.864 INFO:teuthology.orchestra.run.smithi187.stdout:12 Sep 14:51:12 ntpd[27821]: ntp-4 is maintained by Network Time Foundation, 2023-09-12T14:51:12.864 INFO:teuthology.orchestra.run.smithi187.stdout:12 Sep 14:51:12 ntpd[27821]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-09-12T14:51:12.865 INFO:teuthology.orchestra.run.smithi187.stdout:12 Sep 14:51:12 ntpd[27821]: corporation. Support and training for ntp-4 are 2023-09-12T14:51:12.865 INFO:teuthology.orchestra.run.smithi187.stdout:12 Sep 14:51:12 ntpd[27821]: available at https://www.nwtime.org/support 2023-09-12T14:51:12.865 INFO:teuthology.orchestra.run.smithi187.stdout:12 Sep 14:51:12 ntpd[27821]: ---------------------------------------------------- 2023-09-12T14:51:12.866 INFO:teuthology.orchestra.run.smithi187.stdout:12 Sep 14:51:12 ntpd[27821]: proto: precision = 0.063 usec (-24) 2023-09-12T14:51:12.866 INFO:teuthology.orchestra.run.smithi187.stdout:12 Sep 14:51:12 ntpd[27821]: basedate set to 2022-02-04 2023-09-12T14:51:12.866 INFO:teuthology.orchestra.run.smithi187.stdout:12 Sep 14:51:12 ntpd[27821]: gps base set to 2022-02-06 (week 2196) 2023-09-12T14:51:12.866 INFO:teuthology.orchestra.run.smithi187.stdout:12 Sep 14:51:12 ntpd[27821]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-09-12T14:51:12.867 INFO:teuthology.orchestra.run.smithi187.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-09-12T14:51:12.867 INFO:teuthology.orchestra.run.smithi187.stdout:12 Sep 14:51:12 ntpd[27821]: restrict ::: KOD does nothing without LIMITED. 2023-09-12T14:51:12.868 INFO:teuthology.orchestra.run.smithi187.stderr:restrict ::: KOD does nothing without LIMITED. 2023-09-12T14:51:12.869 INFO:teuthology.orchestra.run.smithi187.stdout:12 Sep 14:51:12 ntpd[27821]: Listen and drop on 0 v6wildcard [::]:123 2023-09-12T14:51:12.869 INFO:teuthology.orchestra.run.smithi187.stdout:12 Sep 14:51:12 ntpd[27821]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-09-12T14:51:12.869 INFO:teuthology.orchestra.run.smithi187.stdout:12 Sep 14:51:12 ntpd[27821]: Listen normally on 2 lo 127.0.0.1:123 2023-09-12T14:51:12.869 INFO:teuthology.orchestra.run.smithi187.stdout:12 Sep 14:51:12 ntpd[27821]: Listen normally on 3 enp3s0f1 172.21.15.187:123 2023-09-12T14:51:12.869 INFO:teuthology.orchestra.run.smithi187.stdout:12 Sep 14:51:12 ntpd[27821]: Listen normally on 4 lo [::1]:123 2023-09-12T14:51:12.870 INFO:teuthology.orchestra.run.smithi187.stdout:12 Sep 14:51:12 ntpd[27821]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:77b1%5]:123 2023-09-12T14:51:12.870 INFO:teuthology.orchestra.run.smithi187.stdout:12 Sep 14:51:12 ntpd[27821]: Listening on routing socket on fd #22 for interface updates 2023-09-12T14:51:13.831 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:13 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:13.832 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:13 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:13.832 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:13 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:13.832 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:13 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:13.832 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:13 ntpd[27872]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-12T14:51:13.832 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:13 ntpd[27872]: can't open /var/log/ntpstats/peerstats.20230912: Permission denied 2023-09-12T14:51:13.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:13 ntpd[27821]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:13.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:13 ntpd[27821]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:13.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:13 ntpd[27821]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:13.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:13 ntpd[27821]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:13.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:13 ntpd[27821]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:13.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:13 ntpd[27821]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:13.866 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:13 ntpd[27821]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-12T14:51:13.866 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:13 ntpd[27821]: can't open /var/log/ntpstats/peerstats.20230912: Permission denied 2023-09-12T14:51:13.866 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:13 ntpd[27821]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:13.866 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:13 ntpd[27821]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:13.866 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:13 ntpd[27821]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-12T14:51:13.867 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:13 ntpd[27821]: can't open /var/log/ntpstats/peerstats.20230912: Permission denied 2023-09-12T14:51:14.864 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:14 ntpd[27821]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:14.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:14 ntpd[27821]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:14.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:14 ntpd[27821]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:14.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:14 ntpd[27821]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:14.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:14 ntpd[27821]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-12T14:51:14.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:14 ntpd[27821]: can't open /var/log/ntpstats/peerstats.20230912: Permission denied 2023-09-12T14:51:15.832 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:15 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:15.832 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:15 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:15.832 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:15 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:15.833 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:15 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:15.833 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:15 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:15.833 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:15 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:15.833 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:15 ntpd[27872]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-12T14:51:15.833 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:15 ntpd[27872]: can't open /var/log/ntpstats/peerstats.20230912: Permission denied 2023-09-12T14:51:15.834 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:15 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:15.834 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:15 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:15.834 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:15 ntpd[27872]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-12T14:51:15.834 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:15 ntpd[27872]: can't open /var/log/ntpstats/peerstats.20230912: Permission denied 2023-09-12T14:51:15.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:15 ntpd[27821]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:15.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:15 ntpd[27821]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:15.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:15 ntpd[27821]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:15.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:15 ntpd[27821]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:15.866 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:15 ntpd[27821]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:15.866 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:15 ntpd[27821]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:15.866 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:15 ntpd[27821]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-12T14:51:15.866 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:15 ntpd[27821]: can't open /var/log/ntpstats/peerstats.20230912: Permission denied 2023-09-12T14:51:15.866 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:15 ntpd[27821]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:15.867 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:15 ntpd[27821]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:15.867 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:15 ntpd[27821]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-12T14:51:15.867 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:15 ntpd[27821]: can't open /var/log/ntpstats/peerstats.20230912: Permission denied 2023-09-12T14:51:16.831 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:16 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:16.832 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:16 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:16.832 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:16 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:16.832 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:16 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:16.832 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:16 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:16.833 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:16 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:16.833 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:16 ntpd[27872]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-12T14:51:16.833 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:16 ntpd[27872]: can't open /var/log/ntpstats/peerstats.20230912: Permission denied 2023-09-12T14:51:16.864 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:16 ntpd[27821]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:16.864 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:16 ntpd[27821]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:16.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:16 ntpd[27821]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:16.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:16 ntpd[27821]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:16.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:16 ntpd[27821]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:16.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:16 ntpd[27821]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:16.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:16 ntpd[27821]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-12T14:51:16.866 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:16 ntpd[27821]: can't open /var/log/ntpstats/peerstats.20230912: Permission denied 2023-09-12T14:51:17.832 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:17 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:17.832 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:17 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:17.832 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:17 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:17.833 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:17 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:17.833 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:17 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:17.833 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:17 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:17.833 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:17 ntpd[27872]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-12T14:51:17.833 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:17 ntpd[27872]: can't open /var/log/ntpstats/peerstats.20230912: Permission denied 2023-09-12T14:51:17.834 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:17 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:17.834 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:17 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:17.834 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:17 ntpd[27872]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-12T14:51:17.834 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:17 ntpd[27872]: can't open /var/log/ntpstats/peerstats.20230912: Permission denied 2023-09-12T14:51:17.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:17 ntpd[27821]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:17.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:17 ntpd[27821]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:17.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:17 ntpd[27821]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:17.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:17 ntpd[27821]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:17.866 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:17 ntpd[27821]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:17.866 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:17 ntpd[27821]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:17.866 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:17 ntpd[27821]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-12T14:51:17.866 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:17 ntpd[27821]: can't open /var/log/ntpstats/peerstats.20230912: Permission denied 2023-09-12T14:51:17.866 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:17 ntpd[27821]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:17.867 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:17 ntpd[27821]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:17.867 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:17 ntpd[27821]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-12T14:51:17.867 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:17 ntpd[27821]: can't open /var/log/ntpstats/peerstats.20230912: Permission denied 2023-09-12T14:51:18.831 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:18 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:18.832 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:18 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:18.832 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:18 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:18.832 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:18 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:18.832 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:18 ntpd[27872]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-12T14:51:18.832 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:18 ntpd[27872]: can't open /var/log/ntpstats/peerstats.20230912: Permission denied 2023-09-12T14:51:18.864 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:18 ntpd[27821]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:18.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:18 ntpd[27821]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:18.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:18 ntpd[27821]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:18.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:18 ntpd[27821]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:18.866 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:18 ntpd[27821]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-12T14:51:18.866 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:18 ntpd[27821]: can't open /var/log/ntpstats/peerstats.20230912: Permission denied 2023-09-12T14:51:19.832 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:19 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:19.832 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:19 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:19.832 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:19 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:19.832 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:19 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:19.832 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:19 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:19.833 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:19 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:19.833 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:19 ntpd[27872]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-12T14:51:19.833 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:19 ntpd[27872]: can't open /var/log/ntpstats/peerstats.20230912: Permission denied 2023-09-12T14:51:19.834 INFO:teuthology.orchestra.run.smithi131.stdout:12 Sep 14:51:19 ntpd[27872]: ntpd: time slew -0.006358 s 2023-09-12T14:51:19.834 INFO:teuthology.orchestra.run.smithi131.stdout:ntpd: time slew -0.006358s 2023-09-12T14:51:19.834 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:19 ntpd[27872]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-09-12T14:51:19.835 INFO:teuthology.orchestra.run.smithi131.stderr:12 Sep 14:51:19 ntpd[27872]: can't open /var/log/ntpstats/loopstats.20230912: Permission denied 2023-09-12T14:51:19.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:19 ntpd[27821]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:19.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:19 ntpd[27821]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:19.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:19 ntpd[27821]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:19.865 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:19 ntpd[27821]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:19.866 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:19 ntpd[27821]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-12T14:51:19.866 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:19 ntpd[27821]: can't open /var/log/ntpstats/rawstats.20230912: Permission denied 2023-09-12T14:51:19.866 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:19 ntpd[27821]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-12T14:51:19.866 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:19 ntpd[27821]: can't open /var/log/ntpstats/peerstats.20230912: Permission denied 2023-09-12T14:51:19.867 INFO:teuthology.orchestra.run.smithi187.stdout:12 Sep 14:51:19 ntpd[27821]: ntpd: time slew -0.002596 s 2023-09-12T14:51:19.867 INFO:teuthology.orchestra.run.smithi187.stdout:ntpd: time slew -0.002596s 2023-09-12T14:51:19.868 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:19 ntpd[27821]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-09-12T14:51:19.868 INFO:teuthology.orchestra.run.smithi187.stderr:12 Sep 14:51:19 ntpd[27821]: can't open /var/log/ntpstats/loopstats.20230912: Permission denied 2023-09-12T14:51:19.926 INFO:teuthology.orchestra.run.smithi131.stdout: remote refid st t when poll reach delay offset jitter 2023-09-12T14:51:19.927 INFO:teuthology.orchestra.run.smithi131.stdout:============================================================================== 2023-09-12T14:51:19.927 INFO:teuthology.orchestra.run.smithi131.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-09-12T14:51:19.927 INFO:teuthology.orchestra.run.smithi131.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-09-12T14:51:19.927 INFO:teuthology.orchestra.run.smithi131.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-09-12T14:51:19.927 INFO:teuthology.orchestra.run.smithi131.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-09-12T14:51:19.953 INFO:teuthology.orchestra.run.smithi187.stdout: remote refid st t when poll reach delay offset jitter 2023-09-12T14:51:19.954 INFO:teuthology.orchestra.run.smithi187.stdout:============================================================================== 2023-09-12T14:51:19.954 INFO:teuthology.orchestra.run.smithi187.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-09-12T14:51:19.954 INFO:teuthology.orchestra.run.smithi187.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-09-12T14:51:19.954 INFO:teuthology.orchestra.run.smithi187.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-09-12T14:51:19.955 INFO:teuthology.orchestra.run.smithi187.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-09-12T14:51:19.955 INFO:teuthology.run_tasks:Running task install... 2023-09-12T14:51:19.965 DEBUG:teuthology.task.install:project ceph 2023-09-12T14:51:19.965 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2110e00747b31a0d73768c9e5229da18b26b8aa0'}} 2023-09-12T14:51:19.965 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2110e00747b31a0d73768c9e5229da18b26b8aa0'} 2023-09-12T14:51:19.966 INFO:teuthology.task.install:Using flavor: default 2023-09-12T14:51:19.972 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-09-12T14:51:19.973 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-09-12T14:51:19.973 DEBUG:teuthology.orchestra.run.smithi131:> sudo apt-key list | grep Ceph 2023-09-12T14:51:19.974 DEBUG:teuthology.orchestra.run.smithi187:> sudo apt-key list | grep Ceph 2023-09-12T14:51:20.031 INFO:teuthology.orchestra.run.smithi131.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-09-12T14:51:20.052 INFO:teuthology.orchestra.run.smithi187.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-09-12T14:51:20.057 INFO:teuthology.orchestra.run.smithi131.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-09-12T14:51:20.057 INFO:teuthology.orchestra.run.smithi131.stdout:uid [ unknown] Ceph.com (release key) 2023-09-12T14:51:20.059 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-09-12T14:51:20.059 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-09-12T14:51:20.059 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2110e00747b31a0d73768c9e5229da18b26b8aa0 2023-09-12T14:51:20.078 INFO:teuthology.orchestra.run.smithi187.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-09-12T14:51:20.078 INFO:teuthology.orchestra.run.smithi187.stdout:uid [ unknown] Ceph.com (release key) 2023-09-12T14:51:20.079 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-09-12T14:51:20.080 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-09-12T14:51:20.080 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2110e00747b31a0d73768c9e5229da18b26b8aa0 2023-09-12T14:51:20.332 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default/ 2023-09-12T14:51:20.333 INFO:teuthology.task.install.deb:Package version is 18.0.0-6088-g2110e007-1jammy 2023-09-12T14:51:20.356 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default/ 2023-09-12T14:51:20.356 INFO:teuthology.task.install.deb:Package version is 18.0.0-6088-g2110e007-1jammy 2023-09-12T14:51:20.506 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:51:20.506 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-09-12T14:51:20.521 DEBUG:teuthology.orchestra.run.smithi131:> sudo apt-get update 2023-09-12T14:51:20.535 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:51:20.535 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-09-12T14:51:20.550 DEBUG:teuthology.orchestra.run.smithi187:> sudo apt-get update 2023-09-12T14:51:20.691 INFO:teuthology.orchestra.run.smithi131.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-09-12T14:51:20.699 INFO:teuthology.orchestra.run.smithi131.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-09-12T14:51:20.716 INFO:teuthology.orchestra.run.smithi131.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-09-12T14:51:20.720 INFO:teuthology.orchestra.run.smithi187.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-09-12T14:51:20.745 INFO:teuthology.orchestra.run.smithi131.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy InRelease 2023-09-12T14:51:20.766 INFO:teuthology.orchestra.run.smithi131.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-09-12T14:51:20.788 INFO:teuthology.orchestra.run.smithi131.stdout:Ign:6 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy Release.gpg 2023-09-12T14:51:20.814 INFO:teuthology.orchestra.run.smithi131.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2023-09-12T14:51:20.837 INFO:teuthology.orchestra.run.smithi187.stdout:Ign:2 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy InRelease 2023-09-12T14:51:20.840 INFO:teuthology.orchestra.run.smithi187.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-09-12T14:51:20.852 INFO:teuthology.orchestra.run.smithi131.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,409 B] 2023-09-12T14:51:20.884 INFO:teuthology.orchestra.run.smithi187.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-09-12T14:51:20.922 INFO:teuthology.orchestra.run.smithi187.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-09-12T14:51:20.995 INFO:teuthology.orchestra.run.smithi187.stdout:Ign:6 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy Release.gpg 2023-09-12T14:51:21.004 INFO:teuthology.orchestra.run.smithi187.stdout:Hit:7 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-09-12T14:51:21.065 INFO:teuthology.orchestra.run.smithi187.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2023-09-12T14:51:21.208 INFO:teuthology.orchestra.run.smithi187.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,409 B] 2023-09-12T14:51:21.293 INFO:teuthology.orchestra.run.smithi187.stdout:Fetched 28.6 kB in 1s (51.8 kB/s) 2023-09-12T14:51:22.172 INFO:teuthology.orchestra.run.smithi187.stdout:Reading package lists... 2023-09-12T14:51:22.187 DEBUG:teuthology.orchestra.run.smithi187:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-6088-g2110e007-1jammy cephadm=18.0.0-6088-g2110e007-1jammy ceph-mds=18.0.0-6088-g2110e007-1jammy ceph-mgr=18.0.0-6088-g2110e007-1jammy ceph-common=18.0.0-6088-g2110e007-1jammy ceph-fuse=18.0.0-6088-g2110e007-1jammy ceph-test=18.0.0-6088-g2110e007-1jammy ceph-volume=18.0.0-6088-g2110e007-1jammy radosgw=18.0.0-6088-g2110e007-1jammy python3-rados=18.0.0-6088-g2110e007-1jammy python3-rgw=18.0.0-6088-g2110e007-1jammy python3-cephfs=18.0.0-6088-g2110e007-1jammy python3-rbd=18.0.0-6088-g2110e007-1jammy libcephfs2=18.0.0-6088-g2110e007-1jammy libcephfs-dev=18.0.0-6088-g2110e007-1jammy librados2=18.0.0-6088-g2110e007-1jammy librbd1=18.0.0-6088-g2110e007-1jammy rbd-fuse=18.0.0-6088-g2110e007-1jammy python3-cephfs=18.0.0-6088-g2110e007-1jammy cephfs-shell=18.0.0-6088-g2110e007-1jammy cephfs-top=18.0.0-6088-g2110e007-1jammy cephfs-mirror=18.0.0-6088-g2110e007-1jammy 2023-09-12T14:51:22.247 INFO:teuthology.orchestra.run.smithi187.stdout:Reading package lists... 2023-09-12T14:51:22.430 INFO:teuthology.orchestra.run.smithi187.stdout:Building dependency tree... 2023-09-12T14:51:22.431 INFO:teuthology.orchestra.run.smithi187.stdout:Reading state information... 2023-09-12T14:51:22.590 INFO:teuthology.orchestra.run.smithi187.stdout:The following packages were automatically installed and are no longer required: 2023-09-12T14:51:22.590 INFO:teuthology.orchestra.run.smithi187.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-09-12T14:51:22.590 INFO:teuthology.orchestra.run.smithi187.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-09-12T14:51:22.590 INFO:teuthology.orchestra.run.smithi187.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-09-12T14:51:22.591 INFO:teuthology.orchestra.run.smithi187.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-09-12T14:51:22.592 INFO:teuthology.orchestra.run.smithi187.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-09-12T14:51:22.592 INFO:teuthology.orchestra.run.smithi187.stdout:Use 'sudo apt autoremove' to remove them. 2023-09-12T14:51:22.593 INFO:teuthology.orchestra.run.smithi187.stdout:The following additional packages will be installed: 2023-09-12T14:51:22.593 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-09-12T14:51:22.593 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-09-12T14:51:22.593 INFO:teuthology.orchestra.run.smithi187.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-09-12T14:51:22.593 INFO:teuthology.orchestra.run.smithi187.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-09-12T14:51:22.594 INFO:teuthology.orchestra.run.smithi187.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-09-12T14:51:22.594 INFO:teuthology.orchestra.run.smithi187.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-09-12T14:51:22.594 INFO:teuthology.orchestra.run.smithi187.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-09-12T14:51:22.594 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-09-12T14:51:22.595 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-09-12T14:51:22.595 INFO:teuthology.orchestra.run.smithi187.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-09-12T14:51:22.595 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-09-12T14:51:22.595 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-09-12T14:51:22.595 INFO:teuthology.orchestra.run.smithi187.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-09-12T14:51:22.596 INFO:teuthology.orchestra.run.smithi187.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-09-12T14:51:22.596 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-09-12T14:51:22.596 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-09-12T14:51:22.596 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-09-12T14:51:22.596 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-09-12T14:51:22.597 INFO:teuthology.orchestra.run.smithi187.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-09-12T14:51:22.597 INFO:teuthology.orchestra.run.smithi187.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-09-12T14:51:22.597 INFO:teuthology.orchestra.run.smithi187.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-09-12T14:51:22.597 INFO:teuthology.orchestra.run.smithi187.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-09-12T14:51:22.597 INFO:teuthology.orchestra.run.smithi187.stdout: qttranslations5-l10n socat xmlstarlet 2023-09-12T14:51:22.598 INFO:teuthology.orchestra.run.smithi187.stdout:Suggested packages: 2023-09-12T14:51:22.598 INFO:teuthology.orchestra.run.smithi187.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-09-12T14:51:22.598 INFO:teuthology.orchestra.run.smithi187.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-09-12T14:51:22.599 INFO:teuthology.orchestra.run.smithi187.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-09-12T14:51:22.599 INFO:teuthology.orchestra.run.smithi187.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-09-12T14:51:22.599 INFO:teuthology.orchestra.run.smithi187.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-09-12T14:51:22.599 INFO:teuthology.orchestra.run.smithi187.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-09-12T14:51:22.599 INFO:teuthology.orchestra.run.smithi187.stdout: python3-watchdog 2023-09-12T14:51:22.600 INFO:teuthology.orchestra.run.smithi187.stdout:Recommended packages: 2023-09-12T14:51:22.600 INFO:teuthology.orchestra.run.smithi187.stdout: btrfs-tools 2023-09-12T14:51:22.691 INFO:teuthology.orchestra.run.smithi187.stdout:The following NEW packages will be installed: 2023-09-12T14:51:22.692 INFO:teuthology.orchestra.run.smithi187.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-09-12T14:51:22.692 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-09-12T14:51:22.692 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-09-12T14:51:22.692 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-09-12T14:51:22.693 INFO:teuthology.orchestra.run.smithi187.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-09-12T14:51:22.693 INFO:teuthology.orchestra.run.smithi187.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-09-12T14:51:22.693 INFO:teuthology.orchestra.run.smithi187.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2023-09-12T14:51:22.694 INFO:teuthology.orchestra.run.smithi187.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2023-09-12T14:51:22.695 INFO:teuthology.orchestra.run.smithi187.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-09-12T14:51:22.695 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-09-12T14:51:22.695 INFO:teuthology.orchestra.run.smithi187.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-09-12T14:51:22.695 INFO:teuthology.orchestra.run.smithi187.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-09-12T14:51:22.695 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-09-12T14:51:22.696 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-09-12T14:51:22.696 INFO:teuthology.orchestra.run.smithi187.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-09-12T14:51:22.696 INFO:teuthology.orchestra.run.smithi187.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-09-12T14:51:22.696 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-09-12T14:51:22.696 INFO:teuthology.orchestra.run.smithi187.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-09-12T14:51:22.697 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-09-12T14:51:22.697 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-09-12T14:51:22.697 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-09-12T14:51:22.698 INFO:teuthology.orchestra.run.smithi187.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-09-12T14:51:22.698 INFO:teuthology.orchestra.run.smithi187.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-09-12T14:51:22.698 INFO:teuthology.orchestra.run.smithi187.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-09-12T14:51:22.698 INFO:teuthology.orchestra.run.smithi187.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-09-12T14:51:22.698 INFO:teuthology.orchestra.run.smithi187.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-09-12T14:51:22.699 INFO:teuthology.orchestra.run.smithi187.stdout: socat xmlstarlet 2023-09-12T14:51:22.832 INFO:teuthology.orchestra.run.smithi187.stdout:0 upgraded, 115 newly installed, 0 to remove and 241 not upgraded. 2023-09-12T14:51:22.833 INFO:teuthology.orchestra.run.smithi187.stdout:Need to get 183 MB of archives. 2023-09-12T14:51:22.833 INFO:teuthology.orchestra.run.smithi187.stdout:After this operation, 823 MB of additional disk space will be used. 2023-09-12T14:51:22.833 INFO:teuthology.orchestra.run.smithi187.stdout:Get:1 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-6088-g2110e007-1jammy [3,413 kB] 2023-09-12T14:51:22.878 INFO:teuthology.orchestra.run.smithi187.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-09-12T14:51:22.961 INFO:teuthology.orchestra.run.smithi187.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-09-12T14:51:23.007 INFO:teuthology.orchestra.run.smithi187.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-09-12T14:51:23.053 INFO:teuthology.orchestra.run.smithi187.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-6088-g2110e007-1jammy [3,121 kB] 2023-09-12T14:51:23.147 INFO:teuthology.orchestra.run.smithi187.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-6088-g2110e007-1jammy [670 kB] 2023-09-12T14:51:23.169 INFO:teuthology.orchestra.run.smithi187.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-6088-g2110e007-1jammy [345 kB] 2023-09-12T14:51:23.180 INFO:teuthology.orchestra.run.smithi187.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-6088-g2110e007-1jammy [22.9 kB] 2023-09-12T14:51:23.181 INFO:teuthology.orchestra.run.smithi187.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-6088-g2110e007-1jammy [170 kB] 2023-09-12T14:51:23.186 INFO:teuthology.orchestra.run.smithi187.stdout:Get:10 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-6088-g2110e007-1jammy [52.4 kB] 2023-09-12T14:51:23.188 INFO:teuthology.orchestra.run.smithi187.stdout:Get:11 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-6088-g2110e007-1jammy [318 kB] 2023-09-12T14:51:23.198 INFO:teuthology.orchestra.run.smithi187.stdout:Get:12 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-6088-g2110e007-1jammy [6,112 kB] 2023-09-12T14:51:23.202 INFO:teuthology.orchestra.run.smithi187.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2023-09-12T14:51:23.245 INFO:teuthology.orchestra.run.smithi187.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-09-12T14:51:23.321 INFO:teuthology.orchestra.run.smithi187.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-09-12T14:51:23.332 INFO:teuthology.orchestra.run.smithi187.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-09-12T14:51:23.378 INFO:teuthology.orchestra.run.smithi187.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-09-12T14:51:23.401 INFO:teuthology.orchestra.run.smithi187.stdout:Get:18 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-6088-g2110e007-1jammy [102 kB] 2023-09-12T14:51:23.403 INFO:teuthology.orchestra.run.smithi187.stdout:Get:19 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-6088-g2110e007-1jammy [428 kB] 2023-09-12T14:51:23.416 INFO:teuthology.orchestra.run.smithi187.stdout:Get:20 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-6088-g2110e007-1jammy [22.4 MB] 2023-09-12T14:51:23.545 INFO:teuthology.orchestra.run.smithi187.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-09-12T14:51:23.553 INFO:teuthology.orchestra.run.smithi187.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-09-12T14:51:23.579 INFO:teuthology.orchestra.run.smithi187.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-09-12T14:51:23.751 INFO:teuthology.orchestra.run.smithi187.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-09-12T14:51:23.946 INFO:teuthology.orchestra.run.smithi187.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-09-12T14:51:24.212 INFO:teuthology.orchestra.run.smithi187.stdout:Get:26 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-6088-g2110e007-1jammy [6,820 kB] 2023-09-12T14:51:24.311 INFO:teuthology.orchestra.run.smithi187.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-09-12T14:51:24.319 INFO:teuthology.orchestra.run.smithi187.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-09-12T14:51:24.321 INFO:teuthology.orchestra.run.smithi187.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-09-12T14:51:24.332 INFO:teuthology.orchestra.run.smithi187.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-09-12T14:51:24.333 INFO:teuthology.orchestra.run.smithi187.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-09-12T14:51:24.338 INFO:teuthology.orchestra.run.smithi187.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-09-12T14:51:24.339 INFO:teuthology.orchestra.run.smithi187.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-09-12T14:51:24.340 INFO:teuthology.orchestra.run.smithi187.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-09-12T14:51:24.340 INFO:teuthology.orchestra.run.smithi187.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-09-12T14:51:24.386 INFO:teuthology.orchestra.run.smithi187.stdout:Get:36 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-6088-g2110e007-1jammy [227 kB] 2023-09-12T14:51:24.391 INFO:teuthology.orchestra.run.smithi187.stdout:Get:37 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-6088-g2110e007-1jammy [117 kB] 2023-09-12T14:51:24.395 INFO:teuthology.orchestra.run.smithi187.stdout:Get:38 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-6088-g2110e007-1jammy [1,482 kB] 2023-09-12T14:51:24.444 INFO:teuthology.orchestra.run.smithi187.stdout:Get:39 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-6088-g2110e007-1jammy [9,853 kB] 2023-09-12T14:51:24.506 INFO:teuthology.orchestra.run.smithi187.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-09-12T14:51:24.820 INFO:teuthology.orchestra.run.smithi187.stdout:Get:41 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-6088-g2110e007-1jammy [31.6 MB] 2023-09-12T14:51:24.938 INFO:teuthology.orchestra.run.smithi187.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-09-12T14:51:24.948 INFO:teuthology.orchestra.run.smithi187.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-09-12T14:51:24.953 INFO:teuthology.orchestra.run.smithi187.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-09-12T14:51:24.957 INFO:teuthology.orchestra.run.smithi187.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-09-12T14:51:24.975 INFO:teuthology.orchestra.run.smithi187.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-09-12T14:51:24.979 INFO:teuthology.orchestra.run.smithi187.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-09-12T14:51:24.981 INFO:teuthology.orchestra.run.smithi187.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-09-12T14:51:25.023 INFO:teuthology.orchestra.run.smithi187.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-09-12T14:51:25.037 INFO:teuthology.orchestra.run.smithi187.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-09-12T14:51:25.041 INFO:teuthology.orchestra.run.smithi187.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-09-12T14:51:25.295 INFO:teuthology.orchestra.run.smithi187.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-09-12T14:51:25.378 INFO:teuthology.orchestra.run.smithi187.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-09-12T14:51:25.469 INFO:teuthology.orchestra.run.smithi187.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-09-12T14:51:25.526 INFO:teuthology.orchestra.run.smithi187.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-09-12T14:51:25.647 INFO:teuthology.orchestra.run.smithi187.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-09-12T14:51:25.735 INFO:teuthology.orchestra.run.smithi187.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-09-12T14:51:25.753 INFO:teuthology.orchestra.run.smithi187.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-urllib3 all 1.26.5-1~exp1 [96.4 kB] 2023-09-12T14:51:25.782 INFO:teuthology.orchestra.run.smithi187.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-09-12T14:51:25.795 INFO:teuthology.orchestra.run.smithi187.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-09-12T14:51:25.947 INFO:teuthology.orchestra.run.smithi187.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-09-12T14:51:25.949 INFO:teuthology.orchestra.run.smithi187.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-09-12T14:51:25.949 INFO:teuthology.orchestra.run.smithi187.stdout:Get:63 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-6088-g2110e007-1jammy [3,596 B] 2023-09-12T14:51:25.950 INFO:teuthology.orchestra.run.smithi187.stdout:Get:64 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-6088-g2110e007-1jammy [821 kB] 2023-09-12T14:51:25.967 INFO:teuthology.orchestra.run.smithi187.stdout:Get:65 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-6088-g2110e007-1jammy [2,159 kB] 2023-09-12T14:51:26.004 INFO:teuthology.orchestra.run.smithi187.stdout:Get:66 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-6088-g2110e007-1jammy [245 kB] 2023-09-12T14:51:26.008 INFO:teuthology.orchestra.run.smithi187.stdout:Get:67 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-6088-g2110e007-1jammy [128 kB] 2023-09-12T14:51:26.010 INFO:teuthology.orchestra.run.smithi187.stdout:Get:68 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-6088-g2110e007-1jammy [1,883 kB] 2023-09-12T14:51:26.031 INFO:teuthology.orchestra.run.smithi187.stdout:Get:69 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-6088-g2110e007-1jammy [8,615 kB] 2023-09-12T14:51:26.117 INFO:teuthology.orchestra.run.smithi187.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-09-12T14:51:26.291 INFO:teuthology.orchestra.run.smithi187.stdout:Get:71 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-6088-g2110e007-1jammy [3,726 B] 2023-09-12T14:51:26.292 INFO:teuthology.orchestra.run.smithi187.stdout:Get:72 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-6088-g2110e007-1jammy [47.6 MB] 2023-09-12T14:51:26.929 INFO:teuthology.orchestra.run.smithi187.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-09-12T14:51:26.931 INFO:teuthology.orchestra.run.smithi187.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-09-12T14:51:26.933 INFO:teuthology.orchestra.run.smithi187.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-09-12T14:51:26.935 INFO:teuthology.orchestra.run.smithi187.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-09-12T14:51:26.935 INFO:teuthology.orchestra.run.smithi187.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-09-12T14:51:26.938 INFO:teuthology.orchestra.run.smithi187.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-09-12T14:51:26.979 INFO:teuthology.orchestra.run.smithi187.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-09-12T14:51:26.982 INFO:teuthology.orchestra.run.smithi187.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-09-12T14:51:26.983 INFO:teuthology.orchestra.run.smithi187.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-09-12T14:51:27.040 INFO:teuthology.orchestra.run.smithi187.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-09-12T14:51:27.122 INFO:teuthology.orchestra.run.smithi131.stdout:Hit:9 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-09-12T14:51:27.204 INFO:teuthology.orchestra.run.smithi187.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-09-12T14:51:27.283 INFO:teuthology.orchestra.run.smithi131.stdout:Fetched 28.6 kB in 7s (4,349 B/s) 2023-09-12T14:51:27.463 INFO:teuthology.orchestra.run.smithi187.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-09-12T14:51:27.495 INFO:teuthology.orchestra.run.smithi187.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-09-12T14:51:27.520 INFO:teuthology.orchestra.run.smithi187.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-09-12T14:51:27.572 INFO:teuthology.orchestra.run.smithi187.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-09-12T14:51:27.592 INFO:teuthology.orchestra.run.smithi187.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-09-12T14:51:27.620 INFO:teuthology.orchestra.run.smithi187.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-09-12T14:51:27.748 INFO:teuthology.orchestra.run.smithi187.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-09-12T14:51:27.780 INFO:teuthology.orchestra.run.smithi187.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-09-12T14:51:27.800 INFO:teuthology.orchestra.run.smithi187.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-09-12T14:51:27.807 INFO:teuthology.orchestra.run.smithi187.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-09-12T14:51:28.012 INFO:teuthology.orchestra.run.smithi187.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-09-12T14:51:28.084 INFO:teuthology.orchestra.run.smithi187.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-6088-g2110e007-1jammy [234 kB] 2023-09-12T14:51:28.089 INFO:teuthology.orchestra.run.smithi187.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.0.0-6088-g2110e007-1jammy [215 kB] 2023-09-12T14:51:28.093 INFO:teuthology.orchestra.run.smithi187.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.0.0-6088-g2110e007-1jammy [23.8 kB] 2023-09-12T14:51:28.094 INFO:teuthology.orchestra.run.smithi187.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.0.0-6088-g2110e007-1jammy [13.2 kB] 2023-09-12T14:51:28.095 INFO:teuthology.orchestra.run.smithi187.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-6088-g2110e007-1jammy [28.4 kB] 2023-09-12T14:51:28.096 INFO:teuthology.orchestra.run.smithi187.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-6088-g2110e007-1jammy [12.1 MB] 2023-09-12T14:51:28.132 INFO:teuthology.orchestra.run.smithi131.stdout:Reading package lists... 2023-09-12T14:51:28.148 DEBUG:teuthology.orchestra.run.smithi131:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-6088-g2110e007-1jammy cephadm=18.0.0-6088-g2110e007-1jammy ceph-mds=18.0.0-6088-g2110e007-1jammy ceph-mgr=18.0.0-6088-g2110e007-1jammy ceph-common=18.0.0-6088-g2110e007-1jammy ceph-fuse=18.0.0-6088-g2110e007-1jammy ceph-test=18.0.0-6088-g2110e007-1jammy ceph-volume=18.0.0-6088-g2110e007-1jammy radosgw=18.0.0-6088-g2110e007-1jammy python3-rados=18.0.0-6088-g2110e007-1jammy python3-rgw=18.0.0-6088-g2110e007-1jammy python3-cephfs=18.0.0-6088-g2110e007-1jammy python3-rbd=18.0.0-6088-g2110e007-1jammy libcephfs2=18.0.0-6088-g2110e007-1jammy libcephfs-dev=18.0.0-6088-g2110e007-1jammy librados2=18.0.0-6088-g2110e007-1jammy librbd1=18.0.0-6088-g2110e007-1jammy rbd-fuse=18.0.0-6088-g2110e007-1jammy python3-cephfs=18.0.0-6088-g2110e007-1jammy cephfs-shell=18.0.0-6088-g2110e007-1jammy cephfs-top=18.0.0-6088-g2110e007-1jammy cephfs-mirror=18.0.0-6088-g2110e007-1jammy 2023-09-12T14:51:28.202 INFO:teuthology.orchestra.run.smithi131.stdout:Reading package lists... 2023-09-12T14:51:28.366 INFO:teuthology.orchestra.run.smithi131.stdout:Building dependency tree... 2023-09-12T14:51:28.367 INFO:teuthology.orchestra.run.smithi131.stdout:Reading state information... 2023-09-12T14:51:28.473 INFO:teuthology.orchestra.run.smithi187.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-09-12T14:51:28.478 INFO:teuthology.orchestra.run.smithi187.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-09-12T14:51:28.513 INFO:teuthology.orchestra.run.smithi131.stdout:The following packages were automatically installed and are no longer required: 2023-09-12T14:51:28.513 INFO:teuthology.orchestra.run.smithi131.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-09-12T14:51:28.513 INFO:teuthology.orchestra.run.smithi131.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-09-12T14:51:28.514 INFO:teuthology.orchestra.run.smithi131.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-09-12T14:51:28.514 INFO:teuthology.orchestra.run.smithi131.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-09-12T14:51:28.515 INFO:teuthology.orchestra.run.smithi131.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-09-12T14:51:28.515 INFO:teuthology.orchestra.run.smithi131.stdout:Use 'sudo apt autoremove' to remove them. 2023-09-12T14:51:28.516 INFO:teuthology.orchestra.run.smithi131.stdout:The following additional packages will be installed: 2023-09-12T14:51:28.516 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-09-12T14:51:28.516 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-09-12T14:51:28.516 INFO:teuthology.orchestra.run.smithi131.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-09-12T14:51:28.517 INFO:teuthology.orchestra.run.smithi131.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-09-12T14:51:28.517 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-09-12T14:51:28.517 INFO:teuthology.orchestra.run.smithi131.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-09-12T14:51:28.517 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-09-12T14:51:28.518 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-09-12T14:51:28.518 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-09-12T14:51:28.518 INFO:teuthology.orchestra.run.smithi131.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-09-12T14:51:28.518 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-09-12T14:51:28.519 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-09-12T14:51:28.519 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-09-12T14:51:28.519 INFO:teuthology.orchestra.run.smithi131.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-09-12T14:51:28.519 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-09-12T14:51:28.519 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-09-12T14:51:28.520 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-09-12T14:51:28.520 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-09-12T14:51:28.520 INFO:teuthology.orchestra.run.smithi131.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-09-12T14:51:28.521 INFO:teuthology.orchestra.run.smithi131.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-09-12T14:51:28.521 INFO:teuthology.orchestra.run.smithi131.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-09-12T14:51:28.521 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-09-12T14:51:28.521 INFO:teuthology.orchestra.run.smithi131.stdout: qttranslations5-l10n socat xmlstarlet 2023-09-12T14:51:28.522 INFO:teuthology.orchestra.run.smithi131.stdout:Suggested packages: 2023-09-12T14:51:28.522 INFO:teuthology.orchestra.run.smithi131.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-09-12T14:51:28.522 INFO:teuthology.orchestra.run.smithi131.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-09-12T14:51:28.522 INFO:teuthology.orchestra.run.smithi131.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-09-12T14:51:28.523 INFO:teuthology.orchestra.run.smithi131.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-09-12T14:51:28.523 INFO:teuthology.orchestra.run.smithi131.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-09-12T14:51:28.523 INFO:teuthology.orchestra.run.smithi131.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-09-12T14:51:28.523 INFO:teuthology.orchestra.run.smithi131.stdout: python3-watchdog 2023-09-12T14:51:28.523 INFO:teuthology.orchestra.run.smithi131.stdout:Recommended packages: 2023-09-12T14:51:28.523 INFO:teuthology.orchestra.run.smithi131.stdout: btrfs-tools 2023-09-12T14:51:28.605 INFO:teuthology.orchestra.run.smithi187.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-09-12T14:51:28.616 INFO:teuthology.orchestra.run.smithi131.stdout:The following NEW packages will be installed: 2023-09-12T14:51:28.616 INFO:teuthology.orchestra.run.smithi131.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-09-12T14:51:28.616 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-09-12T14:51:28.617 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-09-12T14:51:28.618 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-09-12T14:51:28.618 INFO:teuthology.orchestra.run.smithi131.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-09-12T14:51:28.619 INFO:teuthology.orchestra.run.smithi131.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-09-12T14:51:28.619 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2023-09-12T14:51:28.619 INFO:teuthology.orchestra.run.smithi131.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2023-09-12T14:51:28.619 INFO:teuthology.orchestra.run.smithi131.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-09-12T14:51:28.619 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-09-12T14:51:28.620 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-09-12T14:51:28.620 INFO:teuthology.orchestra.run.smithi131.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-09-12T14:51:28.620 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-09-12T14:51:28.620 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-09-12T14:51:28.621 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-09-12T14:51:28.621 INFO:teuthology.orchestra.run.smithi131.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-09-12T14:51:28.621 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-09-12T14:51:28.621 INFO:teuthology.orchestra.run.smithi131.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-09-12T14:51:28.622 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-09-12T14:51:28.622 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-09-12T14:51:28.622 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-09-12T14:51:28.622 INFO:teuthology.orchestra.run.smithi131.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-09-12T14:51:28.622 INFO:teuthology.orchestra.run.smithi131.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-09-12T14:51:28.623 INFO:teuthology.orchestra.run.smithi131.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-09-12T14:51:28.623 INFO:teuthology.orchestra.run.smithi131.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-09-12T14:51:28.623 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-09-12T14:51:28.623 INFO:teuthology.orchestra.run.smithi131.stdout: socat xmlstarlet 2023-09-12T14:51:28.653 INFO:teuthology.orchestra.run.smithi187.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-09-12T14:51:28.653 INFO:teuthology.orchestra.run.smithi187.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-09-12T14:51:28.659 INFO:teuthology.orchestra.run.smithi187.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-09-12T14:51:28.661 INFO:teuthology.orchestra.run.smithi187.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-09-12T14:51:28.663 INFO:teuthology.orchestra.run.smithi187.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-6088-g2110e007-1jammy [80.9 kB] 2023-09-12T14:51:28.676 INFO:teuthology.orchestra.run.smithi187.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-09-12T14:51:28.684 INFO:teuthology.orchestra.run.smithi187.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-09-12T14:51:28.745 INFO:teuthology.orchestra.run.smithi187.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-09-12T14:51:28.805 INFO:teuthology.orchestra.run.smithi131.stdout:0 upgraded, 115 newly installed, 0 to remove and 241 not upgraded. 2023-09-12T14:51:28.806 INFO:teuthology.orchestra.run.smithi131.stdout:Need to get 183 MB of archives. 2023-09-12T14:51:28.806 INFO:teuthology.orchestra.run.smithi131.stdout:After this operation, 823 MB of additional disk space will be used. 2023-09-12T14:51:28.806 INFO:teuthology.orchestra.run.smithi131.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-09-12T14:51:28.889 INFO:teuthology.orchestra.run.smithi131.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-09-12T14:51:28.910 INFO:teuthology.orchestra.run.smithi187.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-09-12T14:51:28.919 INFO:teuthology.orchestra.run.smithi131.stdout:Get:3 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-6088-g2110e007-1jammy [3,413 kB] 2023-09-12T14:51:28.933 INFO:teuthology.orchestra.run.smithi131.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-09-12T14:51:29.077 INFO:teuthology.orchestra.run.smithi187.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-09-12T14:51:29.130 INFO:teuthology.orchestra.run.smithi131.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2023-09-12T14:51:29.174 INFO:teuthology.orchestra.run.smithi131.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-09-12T14:51:29.248 INFO:teuthology.orchestra.run.smithi131.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-09-12T14:51:29.260 INFO:teuthology.orchestra.run.smithi131.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-09-12T14:51:29.306 INFO:teuthology.orchestra.run.smithi131.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-09-12T14:51:29.357 INFO:teuthology.orchestra.run.smithi187.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-09-12T14:51:29.382 INFO:teuthology.orchestra.run.smithi187.stdout:Get:115 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-09-12T14:51:29.472 INFO:teuthology.orchestra.run.smithi131.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-09-12T14:51:29.481 INFO:teuthology.orchestra.run.smithi131.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-09-12T14:51:29.508 INFO:teuthology.orchestra.run.smithi131.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-09-12T14:51:29.679 INFO:teuthology.orchestra.run.smithi131.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-09-12T14:51:29.874 INFO:teuthology.orchestra.run.smithi131.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-09-12T14:51:30.064 INFO:teuthology.orchestra.run.smithi131.stdout:Get:15 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-6088-g2110e007-1jammy [3,121 kB] 2023-09-12T14:51:30.183 INFO:teuthology.orchestra.run.smithi187.stdout:Fetched 183 MB in 7s (26.4 MB/s) 2023-09-12T14:51:30.241 INFO:teuthology.orchestra.run.smithi131.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-09-12T14:51:30.248 INFO:teuthology.orchestra.run.smithi131.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-09-12T14:51:30.250 INFO:teuthology.orchestra.run.smithi131.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-09-12T14:51:30.262 INFO:teuthology.orchestra.run.smithi131.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-09-12T14:51:30.262 INFO:teuthology.orchestra.run.smithi131.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-09-12T14:51:30.264 INFO:teuthology.orchestra.run.smithi131.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-09-12T14:51:30.265 INFO:teuthology.orchestra.run.smithi131.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-09-12T14:51:30.268 INFO:teuthology.orchestra.run.smithi131.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-09-12T14:51:30.269 INFO:teuthology.orchestra.run.smithi131.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-09-12T14:51:30.297 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pyperclip. 2023-09-12T14:51:30.388 INFO:teuthology.orchestra.run.smithi187.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 166009 files and directories currently installed.) 2023-09-12T14:51:30.394 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-09-12T14:51:30.435 INFO:teuthology.orchestra.run.smithi131.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-09-12T14:51:30.437 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-09-12T14:51:30.506 INFO:teuthology.orchestra.run.smithi131.stdout:Get:26 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-6088-g2110e007-1jammy [670 kB] 2023-09-12T14:51:30.591 INFO:teuthology.orchestra.run.smithi131.stdout:Get:27 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-6088-g2110e007-1jammy [345 kB] 2023-09-12T14:51:30.651 INFO:teuthology.orchestra.run.smithi131.stdout:Get:28 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-6088-g2110e007-1jammy [22.9 kB] 2023-09-12T14:51:30.653 INFO:teuthology.orchestra.run.smithi131.stdout:Get:29 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-6088-g2110e007-1jammy [170 kB] 2023-09-12T14:51:30.663 INFO:teuthology.orchestra.run.smithi131.stdout:Get:30 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-6088-g2110e007-1jammy [52.4 kB] 2023-09-12T14:51:30.691 INFO:teuthology.orchestra.run.smithi131.stdout:Get:31 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-6088-g2110e007-1jammy [318 kB] 2023-09-12T14:51:30.701 INFO:teuthology.orchestra.run.smithi131.stdout:Get:32 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-6088-g2110e007-1jammy [6,112 kB] 2023-09-12T14:51:30.723 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-wcwidth. 2023-09-12T14:51:30.750 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-09-12T14:51:30.781 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-09-12T14:51:30.867 INFO:teuthology.orchestra.run.smithi131.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-09-12T14:51:30.877 INFO:teuthology.orchestra.run.smithi131.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-09-12T14:51:30.883 INFO:teuthology.orchestra.run.smithi131.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-09-12T14:51:30.887 INFO:teuthology.orchestra.run.smithi131.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-09-12T14:51:30.905 INFO:teuthology.orchestra.run.smithi131.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-09-12T14:51:30.908 INFO:teuthology.orchestra.run.smithi131.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-09-12T14:51:30.910 INFO:teuthology.orchestra.run.smithi131.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-09-12T14:51:30.952 INFO:teuthology.orchestra.run.smithi131.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-09-12T14:51:30.966 INFO:teuthology.orchestra.run.smithi131.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-09-12T14:51:30.971 INFO:teuthology.orchestra.run.smithi131.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-09-12T14:51:31.068 INFO:teuthology.orchestra.run.smithi131.stdout:Get:43 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-6088-g2110e007-1jammy [102 kB] 2023-09-12T14:51:31.082 INFO:teuthology.orchestra.run.smithi131.stdout:Get:44 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-6088-g2110e007-1jammy [428 kB] 2023-09-12T14:51:31.093 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-cmd2. 2023-09-12T14:51:31.093 INFO:teuthology.orchestra.run.smithi131.stdout:Get:45 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-6088-g2110e007-1jammy [22.4 MB] 2023-09-12T14:51:31.119 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-09-12T14:51:31.159 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-09-12T14:51:31.224 INFO:teuthology.orchestra.run.smithi131.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-09-12T14:51:31.307 INFO:teuthology.orchestra.run.smithi131.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-09-12T14:51:31.397 INFO:teuthology.orchestra.run.smithi131.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-09-12T14:51:31.454 INFO:teuthology.orchestra.run.smithi131.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-09-12T14:51:31.479 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libfmt8:amd64. 2023-09-12T14:51:31.505 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2023-09-12T14:51:31.576 INFO:teuthology.orchestra.run.smithi131.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-09-12T14:51:31.587 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2023-09-12T14:51:31.664 INFO:teuthology.orchestra.run.smithi131.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-09-12T14:51:31.682 INFO:teuthology.orchestra.run.smithi131.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-urllib3 all 1.26.5-1~exp1 [96.4 kB] 2023-09-12T14:51:31.711 INFO:teuthology.orchestra.run.smithi131.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-09-12T14:51:31.723 INFO:teuthology.orchestra.run.smithi131.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-09-12T14:51:31.761 INFO:teuthology.orchestra.run.smithi131.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-09-12T14:51:31.769 INFO:teuthology.orchestra.run.smithi131.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-09-12T14:51:31.873 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-09-12T14:51:31.900 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-09-12T14:51:31.931 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-09-12T14:51:31.960 INFO:teuthology.orchestra.run.smithi131.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-09-12T14:51:32.190 INFO:teuthology.orchestra.run.smithi131.stdout:Get:58 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-6088-g2110e007-1jammy [6,820 kB] 2023-09-12T14:51:32.243 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-09-12T14:51:32.270 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-09-12T14:51:32.310 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-09-12T14:51:32.381 INFO:teuthology.orchestra.run.smithi131.stdout:Get:59 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-6088-g2110e007-1jammy [227 kB] 2023-09-12T14:51:32.390 INFO:teuthology.orchestra.run.smithi131.stdout:Get:60 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-6088-g2110e007-1jammy [117 kB] 2023-09-12T14:51:32.396 INFO:teuthology.orchestra.run.smithi131.stdout:Get:61 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-6088-g2110e007-1jammy [1,482 kB] 2023-09-12T14:51:32.454 INFO:teuthology.orchestra.run.smithi131.stdout:Get:62 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-6088-g2110e007-1jammy [9,853 kB] 2023-09-12T14:51:32.688 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-09-12T14:51:32.715 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-09-12T14:51:32.755 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-09-12T14:51:32.770 INFO:teuthology.orchestra.run.smithi131.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-09-12T14:51:32.772 INFO:teuthology.orchestra.run.smithi131.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-09-12T14:51:32.775 INFO:teuthology.orchestra.run.smithi131.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-09-12T14:51:32.776 INFO:teuthology.orchestra.run.smithi131.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-09-12T14:51:32.777 INFO:teuthology.orchestra.run.smithi131.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-09-12T14:51:32.779 INFO:teuthology.orchestra.run.smithi131.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-09-12T14:51:32.823 INFO:teuthology.orchestra.run.smithi131.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-09-12T14:51:32.826 INFO:teuthology.orchestra.run.smithi131.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-09-12T14:51:32.827 INFO:teuthology.orchestra.run.smithi131.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-09-12T14:51:32.833 INFO:teuthology.orchestra.run.smithi131.stdout:Get:72 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-6088-g2110e007-1jammy [31.6 MB] 2023-09-12T14:51:32.880 INFO:teuthology.orchestra.run.smithi131.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-09-12T14:51:33.044 INFO:teuthology.orchestra.run.smithi131.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-09-12T14:51:33.141 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-09-12T14:51:33.168 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-09-12T14:51:33.208 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-09-12T14:51:33.301 INFO:teuthology.orchestra.run.smithi131.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-09-12T14:51:33.335 INFO:teuthology.orchestra.run.smithi131.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-09-12T14:51:33.361 INFO:teuthology.orchestra.run.smithi131.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-09-12T14:51:33.413 INFO:teuthology.orchestra.run.smithi131.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-09-12T14:51:33.432 INFO:teuthology.orchestra.run.smithi131.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-09-12T14:51:33.461 INFO:teuthology.orchestra.run.smithi131.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-09-12T14:51:33.589 INFO:teuthology.orchestra.run.smithi131.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-09-12T14:51:33.619 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-09-12T14:51:33.621 INFO:teuthology.orchestra.run.smithi131.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-09-12T14:51:33.641 INFO:teuthology.orchestra.run.smithi131.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-09-12T14:51:33.646 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-09-12T14:51:33.648 INFO:teuthology.orchestra.run.smithi131.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-09-12T14:51:33.686 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-09-12T14:51:33.852 INFO:teuthology.orchestra.run.smithi131.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-09-12T14:51:34.006 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-09-12T14:51:34.032 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-09-12T14:51:34.072 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-09-12T14:51:34.125 INFO:teuthology.orchestra.run.smithi131.stdout:Get:86 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-6088-g2110e007-1jammy [3,596 B] 2023-09-12T14:51:34.126 INFO:teuthology.orchestra.run.smithi131.stdout:Get:87 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-6088-g2110e007-1jammy [821 kB] 2023-09-12T14:51:34.142 INFO:teuthology.orchestra.run.smithi131.stdout:Get:88 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-6088-g2110e007-1jammy [2,159 kB] 2023-09-12T14:51:34.181 INFO:teuthology.orchestra.run.smithi131.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-6088-g2110e007-1jammy [245 kB] 2023-09-12T14:51:34.186 INFO:teuthology.orchestra.run.smithi131.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-6088-g2110e007-1jammy [128 kB] 2023-09-12T14:51:34.188 INFO:teuthology.orchestra.run.smithi131.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-6088-g2110e007-1jammy [1,883 kB] 2023-09-12T14:51:34.210 INFO:teuthology.orchestra.run.smithi131.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-6088-g2110e007-1jammy [8,615 kB] 2023-09-12T14:51:34.314 INFO:teuthology.orchestra.run.smithi131.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-09-12T14:51:34.319 INFO:teuthology.orchestra.run.smithi131.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-09-12T14:51:34.375 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-09-12T14:51:34.402 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-09-12T14:51:34.434 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-09-12T14:51:34.446 INFO:teuthology.orchestra.run.smithi131.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-09-12T14:51:34.493 INFO:teuthology.orchestra.run.smithi131.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-09-12T14:51:34.494 INFO:teuthology.orchestra.run.smithi131.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-09-12T14:51:34.499 INFO:teuthology.orchestra.run.smithi131.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-09-12T14:51:34.501 INFO:teuthology.orchestra.run.smithi131.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-09-12T14:51:34.513 INFO:teuthology.orchestra.run.smithi131.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-6088-g2110e007-1jammy [3,726 B] 2023-09-12T14:51:34.514 INFO:teuthology.orchestra.run.smithi131.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-6088-g2110e007-1jammy [47.6 MB] 2023-09-12T14:51:34.517 INFO:teuthology.orchestra.run.smithi131.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-09-12T14:51:34.525 INFO:teuthology.orchestra.run.smithi131.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-09-12T14:51:34.585 INFO:teuthology.orchestra.run.smithi131.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-09-12T14:51:34.736 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package librados2. 2023-09-12T14:51:34.750 INFO:teuthology.orchestra.run.smithi131.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-09-12T14:51:34.763 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../011-librados2_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:34.803 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking librados2 (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:34.917 INFO:teuthology.orchestra.run.smithi131.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-09-12T14:51:35.194 INFO:teuthology.orchestra.run.smithi131.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-09-12T14:51:35.219 INFO:teuthology.orchestra.run.smithi131.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-09-12T14:51:35.272 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package librbd1. 2023-09-12T14:51:35.299 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../012-librbd1_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:35.339 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking librbd1 (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:35.733 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libcephfs2. 2023-09-12T14:51:35.757 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../013-libcephfs2_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:35.792 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libcephfs2 (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:36.269 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-rados. 2023-09-12T14:51:36.285 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../014-python3-rados_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:36.327 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-rados (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:36.413 INFO:teuthology.orchestra.run.smithi131.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-6088-g2110e007-1jammy [234 kB] 2023-09-12T14:51:36.493 INFO:teuthology.orchestra.run.smithi131.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.0.0-6088-g2110e007-1jammy [215 kB] 2023-09-12T14:51:36.497 INFO:teuthology.orchestra.run.smithi131.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.0.0-6088-g2110e007-1jammy [23.8 kB] 2023-09-12T14:51:36.498 INFO:teuthology.orchestra.run.smithi131.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.0.0-6088-g2110e007-1jammy [13.2 kB] 2023-09-12T14:51:36.499 INFO:teuthology.orchestra.run.smithi131.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-6088-g2110e007-1jammy [28.4 kB] 2023-09-12T14:51:36.500 INFO:teuthology.orchestra.run.smithi131.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-6088-g2110e007-1jammy [12.1 MB] 2023-09-12T14:51:36.630 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-09-12T14:51:36.656 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.0.0-6088-g2110e007-1jammy_all.deb ... 2023-09-12T14:51:36.697 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-ceph-argparse (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:36.845 INFO:teuthology.orchestra.run.smithi131.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-6088-g2110e007-1jammy [80.9 kB] 2023-09-12T14:51:36.899 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-cephfs. 2023-09-12T14:51:36.914 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../016-python3-cephfs_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:36.957 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-cephfs (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:37.294 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-ceph-common. 2023-09-12T14:51:37.320 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../017-python3-ceph-common_18.0.0-6088-g2110e007-1jammy_all.deb ... 2023-09-12T14:51:37.360 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-ceph-common (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:37.391 INFO:teuthology.orchestra.run.smithi131.stdout:Fetched 183 MB in 8s (22.3 MB/s) 2023-09-12T14:51:37.529 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-pyperclip. 2023-09-12T14:51:37.627 INFO:teuthology.orchestra.run.smithi131.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 166009 files and directories currently installed.) 2023-09-12T14:51:37.633 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-09-12T14:51:37.654 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-prettytable. 2023-09-12T14:51:37.677 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-09-12T14:51:37.681 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2023-09-12T14:51:37.721 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-09-12T14:51:37.972 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-wcwidth. 2023-09-12T14:51:37.974 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-rbd. 2023-09-12T14:51:37.998 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-09-12T14:51:38.000 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../019-python3-rbd_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:38.038 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-09-12T14:51:38.040 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-rbd (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:38.318 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-09-12T14:51:38.345 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-09-12T14:51:38.358 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-cmd2. 2023-09-12T14:51:38.377 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-09-12T14:51:38.385 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-09-12T14:51:38.425 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-09-12T14:51:38.704 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package librgw2. 2023-09-12T14:51:38.731 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../021-librgw2_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:38.753 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libfmt8:amd64. 2023-09-12T14:51:38.771 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking librgw2 (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:38.782 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2023-09-12T14:51:38.861 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2023-09-12T14:51:39.164 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-09-12T14:51:39.191 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-09-12T14:51:39.231 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-09-12T14:51:39.273 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-rgw. 2023-09-12T14:51:39.295 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../022-python3-rgw_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:39.340 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-rgw (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:39.618 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package liboath0:amd64. 2023-09-12T14:51:39.644 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2023-09-12T14:51:39.651 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-09-12T14:51:39.676 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-09-12T14:51:39.678 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-09-12T14:51:39.718 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-09-12T14:51:39.971 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libradosstriper1. 2023-09-12T14:51:39.997 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../024-libradosstriper1_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:40.037 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libradosstriper1 (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:40.055 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-09-12T14:51:40.081 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-09-12T14:51:40.121 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-09-12T14:51:40.457 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-common. 2023-09-12T14:51:40.484 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../025-ceph-common_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:40.523 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-common (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:40.633 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-09-12T14:51:40.660 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-09-12T14:51:40.733 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-09-12T14:51:41.145 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-09-12T14:51:41.172 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-09-12T14:51:41.212 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-09-12T14:51:41.565 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-09-12T14:51:41.592 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-09-12T14:51:41.632 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-09-12T14:51:41.788 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-base. 2023-09-12T14:51:41.814 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../026-ceph-base_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:41.864 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-base (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:42.010 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-09-12T14:51:42.037 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-09-12T14:51:42.085 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-09-12T14:51:42.517 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-09-12T14:51:42.543 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-09-12T14:51:42.583 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-09-12T14:51:42.646 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package librados2. 2023-09-12T14:51:42.673 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../011-librados2_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:42.713 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking librados2 (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:42.853 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-cheroot. 2023-09-12T14:51:42.879 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-09-12T14:51:42.919 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-09-12T14:51:43.166 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package librbd1. 2023-09-12T14:51:43.193 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../012-librbd1_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:43.233 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking librbd1 (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:43.239 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-09-12T14:51:43.266 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-09-12T14:51:43.305 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-09-12T14:51:43.625 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-jaraco.text. 2023-09-12T14:51:43.652 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2023-09-12T14:51:43.708 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-09-12T14:51:43.719 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libcephfs2. 2023-09-12T14:51:43.746 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../013-libcephfs2_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:43.794 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libcephfs2 (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:43.986 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-09-12T14:51:44.001 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-09-12T14:51:44.036 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-09-12T14:51:44.097 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-rados. 2023-09-12T14:51:44.112 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../014-python3-rados_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:44.155 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-rados (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:44.305 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-tempora. 2023-09-12T14:51:44.332 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2023-09-12T14:51:44.372 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-09-12T14:51:44.483 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-09-12T14:51:44.510 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.0.0-6088-g2110e007-1jammy_all.deb ... 2023-09-12T14:51:44.550 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-ceph-argparse (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:44.683 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-portend. 2023-09-12T14:51:44.710 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2023-09-12T14:51:44.750 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-09-12T14:51:44.811 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-cephfs. 2023-09-12T14:51:44.826 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../016-python3-cephfs_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:44.877 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-cephfs (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:45.086 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-09-12T14:51:45.112 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2023-09-12T14:51:45.152 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-09-12T14:51:45.197 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-ceph-common. 2023-09-12T14:51:45.224 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../017-python3-ceph-common_18.0.0-6088-g2110e007-1jammy_all.deb ... 2023-09-12T14:51:45.264 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-ceph-common (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:45.422 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-cherrypy3. 2023-09-12T14:51:45.437 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2023-09-12T14:51:45.471 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-09-12T14:51:45.575 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-prettytable. 2023-09-12T14:51:45.602 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2023-09-12T14:51:45.642 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-09-12T14:51:45.817 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-natsort. 2023-09-12T14:51:45.844 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2023-09-12T14:51:45.883 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-09-12T14:51:45.903 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-rbd. 2023-09-12T14:51:45.918 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../019-python3-rbd_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:45.961 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-rbd (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:46.170 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-logutils. 2023-09-12T14:51:46.185 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2023-09-12T14:51:46.220 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-09-12T14:51:46.381 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-09-12T14:51:46.408 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-09-12T14:51:46.456 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-09-12T14:51:46.706 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-markupsafe. 2023-09-12T14:51:46.732 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-09-12T14:51:46.764 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-09-12T14:51:46.793 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package librgw2. 2023-09-12T14:51:46.819 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../021-librgw2_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:46.859 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking librgw2 (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:47.069 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-mako. 2023-09-12T14:51:47.096 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-09-12T14:51:47.135 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-09-12T14:51:47.456 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-simplegeneric. 2023-09-12T14:51:47.470 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-rgw. 2023-09-12T14:51:47.482 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2023-09-12T14:51:47.497 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../022-python3-rgw_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:47.522 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-09-12T14:51:47.537 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-rgw (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:47.775 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-singledispatch. 2023-09-12T14:51:47.790 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-09-12T14:51:47.825 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-09-12T14:51:47.932 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package liboath0:amd64. 2023-09-12T14:51:47.959 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2023-09-12T14:51:48.006 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-09-12T14:51:48.136 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-webob. 2023-09-12T14:51:48.163 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-09-12T14:51:48.205 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-09-12T14:51:48.301 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libradosstriper1. 2023-09-12T14:51:48.328 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../024-libradosstriper1_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:48.368 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libradosstriper1 (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:48.556 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-waitress. 2023-09-12T14:51:48.583 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-09-12T14:51:48.625 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-09-12T14:51:48.797 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-common. 2023-09-12T14:51:48.823 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../025-ceph-common_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:48.863 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-common (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:48.951 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-tempita. 2023-09-12T14:51:48.977 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-09-12T14:51:49.017 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-09-12T14:51:49.287 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-paste. 2023-09-12T14:51:49.314 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-09-12T14:51:49.345 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-09-12T14:51:49.665 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-09-12T14:51:49.681 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-09-12T14:51:49.723 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-09-12T14:51:49.976 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pastedeploy. 2023-09-12T14:51:49.992 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2023-09-12T14:51:50.034 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-09-12T14:51:50.153 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-base. 2023-09-12T14:51:50.180 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../026-ceph-base_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:50.228 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-base (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:50.338 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-webtest. 2023-09-12T14:51:50.364 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2023-09-12T14:51:50.404 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-09-12T14:51:50.716 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pecan. 2023-09-12T14:51:50.742 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-09-12T14:51:50.791 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-09-12T14:51:50.924 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-09-12T14:51:50.950 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-09-12T14:51:50.991 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-09-12T14:51:51.129 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-certifi. 2023-09-12T14:51:51.156 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2023-09-12T14:51:51.196 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-09-12T14:51:51.294 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-cheroot. 2023-09-12T14:51:51.320 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-09-12T14:51:51.377 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-09-12T14:51:51.532 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-idna. 2023-09-12T14:51:51.547 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2023-09-12T14:51:51.590 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-idna (3.3-1) ... 2023-09-12T14:51:51.672 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-09-12T14:51:51.698 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-09-12T14:51:51.730 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-09-12T14:51:51.885 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-urllib3. 2023-09-12T14:51:51.911 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1_all.deb ... 2023-09-12T14:51:51.951 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1) ... 2023-09-12T14:51:52.033 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-jaraco.text. 2023-09-12T14:51:52.059 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2023-09-12T14:51:52.099 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-09-12T14:51:52.288 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-requests. 2023-09-12T14:51:52.315 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-09-12T14:51:52.355 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-09-12T14:51:52.403 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-09-12T14:51:52.429 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-09-12T14:51:52.567 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-09-12T14:51:52.649 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-werkzeug. 2023-09-12T14:51:52.676 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-09-12T14:51:52.707 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-09-12T14:51:52.906 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-tempora. 2023-09-12T14:51:52.921 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2023-09-12T14:51:52.964 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-09-12T14:51:52.986 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-09-12T14:51:53.002 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.0.0-6088-g2110e007-1jammy_all.deb ... 2023-09-12T14:51:53.036 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-mgr-modules-core (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:53.276 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-portend. 2023-09-12T14:51:53.302 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2023-09-12T14:51:53.342 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-09-12T14:51:53.408 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-09-12T14:51:53.434 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:53.474 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:53.645 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-09-12T14:51:53.671 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2023-09-12T14:51:53.712 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-09-12T14:51:53.777 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-openssl. 2023-09-12T14:51:53.804 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2023-09-12T14:51:53.843 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-09-12T14:51:53.990 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-cherrypy3. 2023-09-12T14:51:54.016 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2023-09-12T14:51:54.056 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-09-12T14:51:54.138 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-mgr. 2023-09-12T14:51:54.165 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../058-ceph-mgr_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:54.205 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-mgr (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:54.410 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-natsort. 2023-09-12T14:51:54.437 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2023-09-12T14:51:54.477 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-09-12T14:51:54.592 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-mon. 2023-09-12T14:51:54.618 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../059-ceph-mon_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:54.658 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-mon (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:54.788 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-logutils. 2023-09-12T14:51:54.815 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2023-09-12T14:51:54.855 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-09-12T14:51:55.125 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-markupsafe. 2023-09-12T14:51:55.151 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-09-12T14:51:55.191 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-09-12T14:51:55.494 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-mako. 2023-09-12T14:51:55.521 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-09-12T14:51:55.561 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-09-12T14:51:55.578 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-osd. 2023-09-12T14:51:55.606 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../060-ceph-osd_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:55.645 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-osd (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:55.948 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-simplegeneric. 2023-09-12T14:51:55.975 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2023-09-12T14:51:56.023 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-09-12T14:51:56.359 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-singledispatch. 2023-09-12T14:51:56.386 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-09-12T14:51:56.426 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-09-12T14:51:56.729 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-webob. 2023-09-12T14:51:56.756 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-09-12T14:51:56.796 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-09-12T14:51:57.191 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-waitress. 2023-09-12T14:51:57.217 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-09-12T14:51:57.268 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-09-12T14:51:57.644 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-tempita. 2023-09-12T14:51:57.670 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-09-12T14:51:57.718 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-09-12T14:51:57.815 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph. 2023-09-12T14:51:57.830 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../061-ceph_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:57.865 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:58.151 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-fuse. 2023-09-12T14:51:58.155 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-paste. 2023-09-12T14:51:58.178 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../062-ceph-fuse_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:58.182 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-09-12T14:51:58.217 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-fuse (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:58.221 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-09-12T14:51:58.575 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-09-12T14:51:58.580 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-mds. 2023-09-12T14:51:58.602 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-09-12T14:51:58.606 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../063-ceph-mds_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:58.641 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-09-12T14:51:58.646 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-mds (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:58.945 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-pastedeploy. 2023-09-12T14:51:58.966 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package cephadm. 2023-09-12T14:51:58.971 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2023-09-12T14:51:58.993 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../064-cephadm_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:51:59.011 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-09-12T14:51:59.024 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking cephadm (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:51:59.314 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-webtest. 2023-09-12T14:51:59.336 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-asyncssh. 2023-09-12T14:51:59.341 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2023-09-12T14:51:59.363 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2023-09-12T14:51:59.398 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-09-12T14:51:59.402 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-09-12T14:51:59.659 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-pecan. 2023-09-12T14:51:59.675 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-09-12T14:51:59.697 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python-babel-localedata. 2023-09-12T14:51:59.712 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-09-12T14:51:59.717 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-09-12T14:51:59.755 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-09-12T14:52:00.057 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-certifi. 2023-09-12T14:52:00.083 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2023-09-12T14:52:00.122 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-09-12T14:52:00.409 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-idna. 2023-09-12T14:52:00.436 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2023-09-12T14:52:00.469 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-idna (3.3-1) ... 2023-09-12T14:52:00.745 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-urllib3. 2023-09-12T14:52:00.771 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1_all.deb ... 2023-09-12T14:52:00.811 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1) ... 2023-09-12T14:52:00.827 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-babel. 2023-09-12T14:52:00.854 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-09-12T14:52:00.894 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-09-12T14:52:01.132 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-requests. 2023-09-12T14:52:01.158 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-09-12T14:52:01.207 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-09-12T14:52:01.222 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-jinja2. 2023-09-12T14:52:01.249 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1_all.deb ... 2023-09-12T14:52:01.289 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-09-12T14:52:01.501 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-werkzeug. 2023-09-12T14:52:01.528 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-09-12T14:52:01.568 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-09-12T14:52:01.650 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-09-12T14:52:01.678 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.0.0-6088-g2110e007-1jammy_all.deb ... 2023-09-12T14:52:01.717 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-mgr-cephadm (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:01.846 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-09-12T14:52:01.861 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.0.0-6088-g2110e007-1jammy_all.deb ... 2023-09-12T14:52:01.896 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-mgr-modules-core (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:02.054 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-bcrypt. 2023-09-12T14:52:02.081 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-09-12T14:52:02.121 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-09-12T14:52:02.268 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-09-12T14:52:02.294 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:52:02.334 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:02.424 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-repoze.lru. 2023-09-12T14:52:02.451 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../071-python3-repoze.lru_0.7-2_all.deb ... 2023-09-12T14:52:02.490 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-09-12T14:52:02.587 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-openssl. 2023-09-12T14:52:02.603 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2023-09-12T14:52:02.637 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-09-12T14:52:02.802 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-routes. 2023-09-12T14:52:02.829 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../072-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-09-12T14:52:02.877 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-09-12T14:52:03.222 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-09-12T14:52:03.249 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../073-ceph-mgr-dashboard_18.0.0-6088-g2110e007-1jammy_all.deb ... 2023-09-12T14:52:03.288 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-mgr-dashboard (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:03.508 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-mgr. 2023-09-12T14:52:03.537 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../058-ceph-mgr_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:52:03.709 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-09-12T14:52:03.736 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-09-12T14:52:03.775 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-09-12T14:52:03.816 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-mgr (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:04.186 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-mon. 2023-09-12T14:52:04.213 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../059-ceph-mon_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:52:04.253 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-mon (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:04.304 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-joblib. 2023-09-12T14:52:04.331 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../075-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-09-12T14:52:04.370 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-09-12T14:52:04.733 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-09-12T14:52:04.760 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../076-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-09-12T14:52:04.799 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-09-12T14:52:05.069 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-sklearn. 2023-09-12T14:52:05.095 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../077-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-09-12T14:52:05.135 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-09-12T14:52:05.165 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-osd. 2023-09-12T14:52:05.192 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../060-ceph-osd_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:52:05.231 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-osd (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:05.778 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-09-12T14:52:05.805 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../078-ceph-mgr-diskprediction-local_18.0.0-6088-g2110e007-1jammy_all.deb ... 2023-09-12T14:52:05.844 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:06.789 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-cachetools. 2023-09-12T14:52:06.816 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../079-python3-cachetools_5.0.0-1_all.deb ... 2023-09-12T14:52:06.855 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-09-12T14:52:07.068 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph. 2023-09-12T14:52:07.084 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../061-ceph_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:52:07.126 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:07.158 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pyasn1. 2023-09-12T14:52:07.185 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../080-python3-pyasn1_0.4.8-1_all.deb ... 2023-09-12T14:52:07.226 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-09-12T14:52:07.388 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-fuse. 2023-09-12T14:52:07.414 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../062-ceph-fuse_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:52:07.454 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-fuse (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:07.729 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-09-12T14:52:07.756 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../081-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-09-12T14:52:07.795 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-09-12T14:52:07.833 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-mds. 2023-09-12T14:52:07.860 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../063-ceph-mds_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:52:07.899 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-mds (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:08.106 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-rsa. 2023-09-12T14:52:08.133 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../082-python3-rsa_4.8-1_all.deb ... 2023-09-12T14:52:08.173 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-rsa (4.8-1) ... 2023-09-12T14:52:08.419 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package cephadm. 2023-09-12T14:52:08.443 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../064-cephadm_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:52:08.478 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking cephadm (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:08.545 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-google-auth. 2023-09-12T14:52:08.572 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../083-python3-google-auth_1.5.1-3_all.deb ... 2023-09-12T14:52:08.612 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-09-12T14:52:08.773 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-asyncssh. 2023-09-12T14:52:08.799 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2023-09-12T14:52:08.839 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-09-12T14:52:08.932 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-09-12T14:52:08.947 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../084-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-09-12T14:52:08.982 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-09-12T14:52:09.159 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python-babel-localedata. 2023-09-12T14:52:09.174 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-09-12T14:52:09.217 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-09-12T14:52:09.235 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-websocket. 2023-09-12T14:52:09.250 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../085-python3-websocket_1.2.3-1_all.deb ... 2023-09-12T14:52:09.293 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-09-12T14:52:09.613 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-kubernetes. 2023-09-12T14:52:09.640 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../086-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-09-12T14:52:09.680 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-09-12T14:52:10.256 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-babel. 2023-09-12T14:52:10.283 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-09-12T14:52:10.319 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-09-12T14:52:10.322 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-09-12T14:52:10.347 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../087-ceph-mgr-k8sevents_18.0.0-6088-g2110e007-1jammy_all.deb ... 2023-09-12T14:52:10.386 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:10.643 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-jinja2. 2023-09-12T14:52:10.670 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1_all.deb ... 2023-09-12T14:52:10.705 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libonig5:amd64. 2023-09-12T14:52:10.709 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-09-12T14:52:10.732 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../088-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-09-12T14:52:10.772 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-09-12T14:52:11.055 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-09-12T14:52:11.075 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libjq1:amd64. 2023-09-12T14:52:11.081 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.0.0-6088-g2110e007-1jammy_all.deb ... 2023-09-12T14:52:11.102 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../089-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-09-12T14:52:11.121 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-mgr-cephadm (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:11.141 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-09-12T14:52:11.369 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package jq. 2023-09-12T14:52:11.393 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../090-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-09-12T14:52:11.444 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-09-12T14:52:11.467 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-bcrypt. 2023-09-12T14:52:11.493 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-09-12T14:52:11.533 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-09-12T14:52:11.688 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package socat. 2023-09-12T14:52:11.704 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../091-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-09-12T14:52:11.747 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-09-12T14:52:11.836 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-repoze.lru. 2023-09-12T14:52:11.863 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../071-python3-repoze.lru_0.7-2_all.deb ... 2023-09-12T14:52:11.902 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-09-12T14:52:12.075 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package xmlstarlet. 2023-09-12T14:52:12.102 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../092-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-09-12T14:52:12.141 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-09-12T14:52:12.164 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-routes. 2023-09-12T14:52:12.181 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../072-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-09-12T14:52:12.222 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-09-12T14:52:12.471 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-test. 2023-09-12T14:52:12.499 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../093-ceph-test_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:52:12.529 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-test (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:12.543 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-09-12T14:52:12.570 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../073-ceph-mgr-dashboard_18.0.0-6088-g2110e007-1jammy_all.deb ... 2023-09-12T14:52:12.609 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-mgr-dashboard (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:13.055 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-09-12T14:52:13.070 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-09-12T14:52:13.104 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-09-12T14:52:13.559 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-joblib. 2023-09-12T14:52:13.585 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../075-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-09-12T14:52:13.625 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-09-12T14:52:13.995 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-09-12T14:52:14.022 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../076-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-09-12T14:52:14.062 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-09-12T14:52:14.357 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-sklearn. 2023-09-12T14:52:14.384 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../077-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-09-12T14:52:14.415 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-09-12T14:52:14.708 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-volume. 2023-09-12T14:52:14.736 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../094-ceph-volume_18.0.0-6088-g2110e007-1jammy_all.deb ... 2023-09-12T14:52:14.775 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-volume (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:15.108 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-09-12T14:52:15.135 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../078-ceph-mgr-diskprediction-local_18.0.0-6088-g2110e007-1jammy_all.deb ... 2023-09-12T14:52:15.164 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package cephfs-mirror. 2023-09-12T14:52:15.176 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:15.191 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../095-cephfs-mirror_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:52:15.230 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking cephfs-mirror (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:15.509 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-colorama. 2023-09-12T14:52:15.536 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../096-python3-colorama_0.4.4-1_all.deb ... 2023-09-12T14:52:15.574 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-09-12T14:52:15.844 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package cephfs-shell. 2023-09-12T14:52:15.861 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../097-cephfs-shell_18.0.0-6088-g2110e007-1jammy_all.deb ... 2023-09-12T14:52:15.902 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking cephfs-shell (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:16.155 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package cephfs-top. 2023-09-12T14:52:16.171 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../098-cephfs-top_18.0.0-6088-g2110e007-1jammy_all.deb ... 2023-09-12T14:52:16.204 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-cachetools. 2023-09-12T14:52:16.214 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking cephfs-top (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:16.231 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../079-python3-cachetools_5.0.0-1_all.deb ... 2023-09-12T14:52:16.271 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-09-12T14:52:16.441 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libcephfs-dev. 2023-09-12T14:52:16.457 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../099-libcephfs-dev_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:52:16.491 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libcephfs-dev (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:16.507 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-pyasn1. 2023-09-12T14:52:16.523 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../080-python3-pyasn1_0.4.8-1_all.deb ... 2023-09-12T14:52:16.557 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-09-12T14:52:16.862 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package nvme-cli. 2023-09-12T14:52:16.889 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../100-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-09-12T14:52:16.894 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-09-12T14:52:16.921 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../081-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-09-12T14:52:16.928 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-09-12T14:52:16.960 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-09-12T14:52:17.255 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-rsa. 2023-09-12T14:52:17.282 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../082-python3-rsa_4.8-1_all.deb ... 2023-09-12T14:52:17.358 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-09-12T14:52:17.374 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../101-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-09-12T14:52:17.411 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-rsa (4.8-1) ... 2023-09-12T14:52:17.416 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-09-12T14:52:17.836 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-iniconfig. 2023-09-12T14:52:17.863 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../102-python3-iniconfig_1.1.1-2_all.deb ... 2023-09-12T14:52:17.902 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-google-auth. 2023-09-12T14:52:17.903 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-09-12T14:52:17.929 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../083-python3-google-auth_1.5.1-3_all.deb ... 2023-09-12T14:52:17.969 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-09-12T14:52:18.248 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-09-12T14:52:18.274 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../084-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-09-12T14:52:18.305 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-09-12T14:52:18.364 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pastescript. 2023-09-12T14:52:18.392 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-4_all.deb ... 2023-09-12T14:52:18.430 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-09-12T14:52:18.592 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-websocket. 2023-09-12T14:52:18.619 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../085-python3-websocket_1.2.3-1_all.deb ... 2023-09-12T14:52:18.659 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-09-12T14:52:18.726 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pluggy. 2023-09-12T14:52:18.753 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../104-python3-pluggy_0.13.0-7.1_all.deb ... 2023-09-12T14:52:18.792 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-09-12T14:52:18.987 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-kubernetes. 2023-09-12T14:52:19.014 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../086-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-09-12T14:52:19.054 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-09-12T14:52:19.095 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-psutil. 2023-09-12T14:52:19.123 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../105-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-09-12T14:52:19.162 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-09-12T14:52:19.507 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-py. 2023-09-12T14:52:19.534 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../106-python3-py_1.10.0-1_all.deb ... 2023-09-12T14:52:19.573 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-py (1.10.0-1) ... 2023-09-12T14:52:19.726 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-09-12T14:52:19.743 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../087-ceph-mgr-k8sevents_18.0.0-6088-g2110e007-1jammy_all.deb ... 2023-09-12T14:52:19.784 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:19.902 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pygments. 2023-09-12T14:52:19.929 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../107-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-09-12T14:52:19.968 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-09-12T14:52:20.071 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libonig5:amd64. 2023-09-12T14:52:20.099 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../088-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-09-12T14:52:20.138 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-09-12T14:52:20.406 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pyinotify. 2023-09-12T14:52:20.422 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../108-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-09-12T14:52:20.464 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-09-12T14:52:20.465 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libjq1:amd64. 2023-09-12T14:52:20.493 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../089-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-09-12T14:52:20.532 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-09-12T14:52:20.742 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-toml. 2023-09-12T14:52:20.769 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../109-python3-toml_0.10.2-1_all.deb ... 2023-09-12T14:52:20.785 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package jq. 2023-09-12T14:52:20.801 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../090-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-09-12T14:52:20.808 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-09-12T14:52:20.835 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-09-12T14:52:21.046 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package socat. 2023-09-12T14:52:21.062 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../091-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-09-12T14:52:21.094 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pytest. 2023-09-12T14:52:21.104 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-09-12T14:52:21.122 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../110-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-09-12T14:52:21.169 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-09-12T14:52:21.399 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package xmlstarlet. 2023-09-12T14:52:21.415 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../092-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-09-12T14:52:21.449 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-09-12T14:52:21.473 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-simplejson. 2023-09-12T14:52:21.501 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-09-12T14:52:21.539 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-09-12T14:52:21.796 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-test. 2023-09-12T14:52:21.800 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-09-12T14:52:21.823 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../093-ceph-test_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:52:21.828 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-09-12T14:52:21.862 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-test (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:21.867 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-09-12T14:52:22.563 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package radosgw. 2023-09-12T14:52:22.591 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../113-radosgw_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:52:22.629 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking radosgw (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:23.399 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package rbd-fuse. 2023-09-12T14:52:23.427 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../114-rbd-fuse_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:52:23.466 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking rbd-fuse (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:23.754 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-09-12T14:52:24.012 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-09-12T14:52:24.138 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-09-12T14:52:24.216 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-volume. 2023-09-12T14:52:24.232 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../094-ceph-volume_18.0.0-6088-g2110e007-1jammy_all.deb ... 2023-09-12T14:52:24.274 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-volume (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:24.395 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-09-12T14:52:24.746 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-09-12T14:52:24.763 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package cephfs-mirror. 2023-09-12T14:52:24.790 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../095-cephfs-mirror_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:52:24.838 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking cephfs-mirror (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:25.158 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-colorama. 2023-09-12T14:52:25.185 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../096-python3-colorama_0.4.4-1_all.deb ... 2023-09-12T14:52:25.224 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-09-12T14:52:25.249 INFO:teuthology.orchestra.run.smithi187.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-09-12T14:52:25.324 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up cephadm (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:25.486 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package cephfs-shell. 2023-09-12T14:52:25.502 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../097-cephfs-shell_18.0.0-6088-g2110e007-1jammy_all.deb ... 2023-09-12T14:52:25.544 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking cephfs-shell (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:25.781 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package cephfs-top. 2023-09-12T14:52:25.809 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../098-cephfs-top_18.0.0-6088-g2110e007-1jammy_all.deb ... 2023-09-12T14:52:25.839 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking cephfs-top (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:26.117 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libcephfs-dev. 2023-09-12T14:52:26.132 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../099-libcephfs-dev_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:52:26.154 INFO:teuthology.orchestra.run.smithi187.stdout:Adding system user cephadm....done 2023-09-12T14:52:26.175 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libcephfs-dev (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:26.283 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-09-12T14:52:26.551 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-09-12T14:52:26.570 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package nvme-cli. 2023-09-12T14:52:26.598 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../100-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-09-12T14:52:26.637 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-09-12T14:52:26.794 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-09-12T14:52:26.920 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-09-12T14:52:26.984 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-09-12T14:52:27.011 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../101-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-09-12T14:52:27.050 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-09-12T14:52:27.196 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-09-12T14:52:27.472 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-09-12T14:52:27.479 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-iniconfig. 2023-09-12T14:52:27.506 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../102-python3-iniconfig_1.1.1-2_all.deb ... 2023-09-12T14:52:27.546 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-09-12T14:52:27.598 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-py (1.10.0-1) ... 2023-09-12T14:52:27.865 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-pastescript. 2023-09-12T14:52:27.892 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-4_all.deb ... 2023-09-12T14:52:27.900 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-09-12T14:52:27.931 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-09-12T14:52:28.252 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-pluggy. 2023-09-12T14:52:28.279 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../104-python3-pluggy_0.13.0-7.1_all.deb ... 2023-09-12T14:52:28.326 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-09-12T14:52:28.410 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-09-12T14:52:28.677 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-09-12T14:52:28.705 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-psutil. 2023-09-12T14:52:28.733 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../105-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-09-12T14:52:28.780 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-09-12T14:52:28.928 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-09-12T14:52:29.083 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-py. 2023-09-12T14:52:29.110 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../106-python3-py_1.10.0-1_all.deb ... 2023-09-12T14:52:29.149 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-py (1.10.0-1) ... 2023-09-12T14:52:29.205 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-09-12T14:52:29.486 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-09-12T14:52:29.503 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-pygments. 2023-09-12T14:52:29.531 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../107-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-09-12T14:52:29.570 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-09-12T14:52:29.773 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-09-12T14:52:30.024 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-ceph-argparse (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:30.066 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-pyinotify. 2023-09-12T14:52:30.093 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../108-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-09-12T14:52:30.132 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-09-12T14:52:30.267 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-09-12T14:52:30.385 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-09-12T14:52:30.443 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-toml. 2023-09-12T14:52:30.471 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../109-python3-toml_0.10.2-1_all.deb ... 2023-09-12T14:52:30.510 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-09-12T14:52:30.653 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-09-12T14:52:30.779 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-pytest. 2023-09-12T14:52:30.795 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../110-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-09-12T14:52:30.829 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-09-12T14:52:30.912 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-09-12T14:52:31.072 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-09-12T14:52:31.175 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-simplejson. 2023-09-12T14:52:31.203 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-09-12T14:52:31.241 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-09-12T14:52:31.439 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-09-12T14:52:31.519 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-09-12T14:52:31.544 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-09-12T14:52:31.557 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-09-12T14:52:31.586 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-09-12T14:52:31.800 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-09-12T14:52:32.034 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-09-12T14:52:32.344 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-09-12T14:52:32.349 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package radosgw. 2023-09-12T14:52:32.376 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../113-radosgw_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:52:32.415 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking radosgw (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:32.980 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-09-12T14:52:33.202 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package rbd-fuse. 2023-09-12T14:52:33.217 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../114-rbd-fuse_18.0.0-6088-g2110e007-1jammy_amd64.deb ... 2023-09-12T14:52:33.247 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-09-12T14:52:33.259 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking rbd-fuse (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:33.373 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-09-12T14:52:33.491 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-09-12T14:52:33.547 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-09-12T14:52:33.792 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-09-12T14:52:33.798 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-09-12T14:52:33.924 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-09-12T14:52:34.143 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-09-12T14:52:34.173 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-09-12T14:52:34.385 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-09-12T14:52:34.525 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-09-12T14:52:34.762 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-09-12T14:52:34.979 INFO:teuthology.orchestra.run.smithi131.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-09-12T14:52:35.076 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up cephadm (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:35.080 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-09-12T14:52:35.381 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-09-12T14:52:35.511 INFO:teuthology.orchestra.run.smithi131.stdout:Adding system user cephadm....done 2023-09-12T14:52:35.666 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-09-12T14:52:35.699 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-09-12T14:52:35.925 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-09-12T14:52:35.934 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-09-12T14:52:36.051 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-ceph-common (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:36.186 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-09-12T14:52:36.303 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-09-12T14:52:36.336 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-idna (3.3-1) ... 2023-09-12T14:52:36.554 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-09-12T14:52:36.654 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-09-12T14:52:36.806 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-09-12T14:52:36.932 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-py (1.10.0-1) ... 2023-09-12T14:52:37.244 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-09-12T14:52:37.578 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-09-12T14:52:37.818 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-09-12T14:52:37.837 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-09-12T14:52:37.952 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-toml (0.10.2-1) ... 2023-09-12T14:52:38.089 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-09-12T14:52:38.220 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-09-12T14:52:38.346 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-09-12T14:52:38.382 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-09-12T14:52:38.605 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-urllib3 (1.26.5-1~exp1) ... 2023-09-12T14:52:38.633 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-09-12T14:52:38.901 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-09-12T14:52:38.940 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-09-12T14:52:39.143 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-ceph-argparse (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:39.183 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-09-12T14:52:39.409 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-09-12T14:52:39.535 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2023-09-12T14:52:39.661 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-09-12T14:52:39.728 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-09-12T14:52:39.854 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-09-12T14:52:39.954 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-09-12T14:52:40.114 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-09-12T14:52:40.222 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-09-12T14:52:40.365 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-09-12T14:52:40.481 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-09-12T14:52:40.490 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-09-12T14:52:40.724 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-09-12T14:52:40.842 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-09-12T14:52:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-09-12T14:52:40.966 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-09-12T14:52:41.202 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-09-12T14:52:41.234 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-09-12T14:52:41.360 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-09-12T14:52:41.454 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-09-12T14:52:41.645 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-09-12T14:52:41.772 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-09-12T14:52:41.779 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-09-12T14:52:42.072 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-09-12T14:52:42.365 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-09-12T14:52:42.449 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-09-12T14:52:42.591 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-09-12T14:52:42.708 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-09-12T14:52:42.818 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-09-12T14:52:42.936 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-09-12T14:52:42.991 INFO:teuthology.orchestra.run.smithi187.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-09-12T14:52:43.212 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-09-12T14:52:43.252 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-09-12T14:52:43.378 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-09-12T14:52:43.588 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-09-12T14:52:43.822 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-09-12T14:52:43.839 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-09-12T14:52:44.081 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-portend (3.0.0-1) ... 2023-09-12T14:52:44.233 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-09-12T14:52:44.315 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-09-12T14:52:44.425 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-09-12T14:52:44.517 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-09-12T14:52:44.751 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-09-12T14:52:44.802 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-09-12T14:52:45.069 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-09-12T14:52:45.129 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-09-12T14:52:45.345 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-09-12T14:52:45.396 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-09-12T14:52:45.488 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-rsa (4.8-1) ... 2023-09-12T14:52:45.522 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-ceph-common (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:45.763 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-09-12T14:52:45.824 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-idna (3.3-1) ... 2023-09-12T14:52:46.023 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-09-12T14:52:46.133 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-09-12T14:52:46.266 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-09-12T14:52:46.676 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-09-12T14:52:46.968 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-09-12T14:52:47.240 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-09-12T14:52:47.312 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-09-12T14:52:47.374 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-toml (0.10.2-1) ... 2023-09-12T14:52:47.438 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up librados2 (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:47.556 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up librgw2 (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:47.633 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-09-12T14:52:47.734 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-09-12T14:52:47.751 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libsqlite3-mod-ceph (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:47.844 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-urllib3 (1.26.5-1~exp1) ... 2023-09-12T14:52:47.877 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libcephfs2 (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:48.003 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libradosstriper1 (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:48.137 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-09-12T14:52:48.138 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-09-12T14:52:48.397 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-09-12T14:52:48.414 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up librbd1 (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:48.532 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-mgr-modules-core (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:48.648 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-09-12T14:52:48.649 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-fuse (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:48.774 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2023-09-12T14:52:48.853 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-09-12T14:52:48.854 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-09-12T14:52:48.900 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-09-12T14:52:49.193 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-09-12T14:52:49.401 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libcephfs-dev (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:49.444 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-09-12T14:52:49.526 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-rados (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:49.652 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up cephfs-top (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:49.704 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-09-12T14:52:49.787 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-09-12T14:52:49.972 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-09-12T14:52:50.231 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-09-12T14:52:50.499 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-09-12T14:52:50.625 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-09-12T14:52:50.893 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-09-12T14:52:50.994 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-09-12T14:52:51.008 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-rbd (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:51.134 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up rbd-fuse (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:51.260 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-rgw (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:51.287 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-09-12T14:52:51.386 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-cephfs (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:51.512 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up cephfs-shell (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:51.580 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-09-12T14:52:51.638 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-common (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:51.831 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-09-12T14:52:52.156 INFO:teuthology.orchestra.run.smithi187.stdout:Adding group ceph....done 2023-09-12T14:52:52.213 INFO:teuthology.orchestra.run.smithi131.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-09-12T14:52:52.492 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-09-12T14:52:52.540 INFO:teuthology.orchestra.run.smithi187.stdout:Adding system user ceph....done 2023-09-12T14:52:52.610 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-09-12T14:52:52.700 INFO:teuthology.orchestra.run.smithi187.stdout:Setting system user ceph properties....done 2023-09-12T14:52:52.715 INFO:teuthology.orchestra.run.smithi187.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-09-12T14:52:52.847 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-09-12T14:52:53.029 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-09-12T14:52:53.225 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-09-12T14:52:53.288 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-portend (3.0.0-1) ... 2023-09-12T14:52:53.564 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-09-12T14:52:53.690 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-09-12T14:52:53.748 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up cephfs-mirror (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:53.947 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-09-12T14:52:53.947 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-09-12T14:52:54.008 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-09-12T14:52:54.292 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-09-12T14:52:54.457 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-test (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:54.552 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-09-12T14:52:54.575 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up radosgw (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:54.678 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-rsa (4.8-1) ... 2023-09-12T14:52:54.812 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-09-12T14:52:54.812 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-09-12T14:52:55.128 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-09-12T14:52:55.464 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-09-12T14:52:55.543 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-base (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:55.723 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-09-12T14:52:55.888 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-09-12T14:52:56.108 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-09-12T14:52:56.368 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-09-12T14:52:56.404 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-mds (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:56.621 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-09-12T14:52:56.622 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-09-12T14:52:56.687 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-09-12T14:52:56.838 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up librados2 (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:56.964 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up librgw2 (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:57.122 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-mgr (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:57.159 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libsqlite3-mod-ceph (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:57.261 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libcephfs2 (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:57.328 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-09-12T14:52:57.328 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-09-12T14:52:57.378 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libradosstriper1 (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:57.505 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-09-12T14:52:57.781 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up librbd1 (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:57.798 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-osd (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:57.907 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-mgr-modules-core (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:58.034 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-fuse (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:58.143 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-09-12T14:52:58.144 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-09-12T14:52:58.253 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-09-12T14:52:58.253 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-09-12T14:52:58.691 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-mgr-k8sevents (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:58.777 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libcephfs-dev (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:58.817 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:58.945 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-rados (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:58.994 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-mon (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:59.088 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up cephfs-top (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:59.215 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-09-12T14:52:59.215 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-09-12T14:52:59.241 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-09-12T14:52:59.745 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-mgr-cephadm (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:52:59.880 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:53:00.005 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-mgr-dashboard (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:53:00.165 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-volume (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:53:00.292 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-09-12T14:53:00.606 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-rbd (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:53:00.833 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up rbd-fuse (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:53:01.061 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-rgw (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:53:01.255 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-cephfs (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:53:01.367 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-09-12T14:53:01.381 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up cephfs-shell (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:53:01.507 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-common (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:53:01.842 INFO:teuthology.orchestra.run.smithi187.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-09-12T14:53:01.845 DEBUG:teuthology.orchestra.run.smithi187:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-09-12T14:53:01.899 INFO:teuthology.orchestra.run.smithi187.stdout:Reading package lists... 2023-09-12T14:53:01.999 INFO:teuthology.orchestra.run.smithi131.stdout:Adding group ceph....done 2023-09-12T14:53:02.069 INFO:teuthology.orchestra.run.smithi187.stdout:Building dependency tree... 2023-09-12T14:53:02.070 INFO:teuthology.orchestra.run.smithi187.stdout:Reading state information... 2023-09-12T14:53:02.215 INFO:teuthology.orchestra.run.smithi187.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-09-12T14:53:02.215 INFO:teuthology.orchestra.run.smithi187.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-09-12T14:53:02.215 INFO:teuthology.orchestra.run.smithi187.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-09-12T14:53:02.216 INFO:teuthology.orchestra.run.smithi187.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-09-12T14:53:02.216 INFO:teuthology.orchestra.run.smithi187.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-09-12T14:53:02.216 INFO:teuthology.orchestra.run.smithi187.stdout:The following packages were automatically installed and are no longer required: 2023-09-12T14:53:02.216 INFO:teuthology.orchestra.run.smithi187.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-09-12T14:53:02.217 INFO:teuthology.orchestra.run.smithi187.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-09-12T14:53:02.217 INFO:teuthology.orchestra.run.smithi187.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-09-12T14:53:02.217 INFO:teuthology.orchestra.run.smithi187.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-09-12T14:53:02.217 INFO:teuthology.orchestra.run.smithi187.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-09-12T14:53:02.217 INFO:teuthology.orchestra.run.smithi187.stdout:Use 'sudo apt autoremove' to remove them. 2023-09-12T14:53:02.218 INFO:teuthology.orchestra.run.smithi187.stdout:The following additional packages will be installed: 2023-09-12T14:53:02.219 INFO:teuthology.orchestra.run.smithi187.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-09-12T14:53:02.219 INFO:teuthology.orchestra.run.smithi187.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-09-12T14:53:02.219 INFO:teuthology.orchestra.run.smithi187.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-09-12T14:53:02.219 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-09-12T14:53:02.219 INFO:teuthology.orchestra.run.smithi187.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-09-12T14:53:02.220 INFO:teuthology.orchestra.run.smithi187.stdout:Suggested packages: 2023-09-12T14:53:02.220 INFO:teuthology.orchestra.run.smithi187.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-09-12T14:53:02.220 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-09-12T14:53:02.220 INFO:teuthology.orchestra.run.smithi187.stdout: libengine-pkcs11-openssl 2023-09-12T14:53:02.251 INFO:teuthology.orchestra.run.smithi187.stdout:The following NEW packages will be installed: 2023-09-12T14:53:02.253 INFO:teuthology.orchestra.run.smithi187.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-09-12T14:53:02.253 INFO:teuthology.orchestra.run.smithi187.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-09-12T14:53:02.253 INFO:teuthology.orchestra.run.smithi187.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-09-12T14:53:02.253 INFO:teuthology.orchestra.run.smithi187.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-09-12T14:53:02.253 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-09-12T14:53:02.254 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-09-12T14:53:02.254 INFO:teuthology.orchestra.run.smithi187.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-09-12T14:53:02.254 INFO:teuthology.orchestra.run.smithi187.stdout:The following packages will be upgraded: 2023-09-12T14:53:02.255 INFO:teuthology.orchestra.run.smithi187.stdout: libssl3 2023-09-12T14:53:02.327 INFO:teuthology.orchestra.run.smithi187.stdout:1 upgraded, 39 newly installed, 0 to remove and 240 not upgraded. 2023-09-12T14:53:02.328 INFO:teuthology.orchestra.run.smithi187.stdout:Need to get 54.6 MB of archives. 2023-09-12T14:53:02.328 INFO:teuthology.orchestra.run.smithi187.stdout:After this operation, 202 MB of additional disk space will be used. 2023-09-12T14:53:02.328 INFO:teuthology.orchestra.run.smithi187.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-09-12T14:53:02.375 INFO:teuthology.orchestra.run.smithi131.stdout:Adding system user ceph....done 2023-09-12T14:53:02.431 INFO:teuthology.orchestra.run.smithi187.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.10 [1,901 kB] 2023-09-12T14:53:02.492 INFO:teuthology.orchestra.run.smithi187.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-09-12T14:53:02.507 INFO:teuthology.orchestra.run.smithi187.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-09-12T14:53:02.508 INFO:teuthology.orchestra.run.smithi187.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-09-12T14:53:02.510 INFO:teuthology.orchestra.run.smithi187.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 indent amd64 2.2.12-1build2 [90.8 kB] 2023-09-12T14:53:02.512 INFO:teuthology.orchestra.run.smithi187.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-09-12T14:53:02.512 INFO:teuthology.orchestra.run.smithi187.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-09-12T14:53:02.514 INFO:teuthology.orchestra.run.smithi187.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-09-12T14:53:02.517 INFO:teuthology.orchestra.run.smithi187.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-09-12T14:53:02.518 INFO:teuthology.orchestra.run.smithi187.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-09-12T14:53:02.518 INFO:teuthology.orchestra.run.smithi187.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-09-12T14:53:02.532 INFO:teuthology.orchestra.run.smithi187.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-09-12T14:53:02.551 INFO:teuthology.orchestra.run.smithi131.stdout:Setting system user ceph properties....done 2023-09-12T14:53:02.567 INFO:teuthology.orchestra.run.smithi131.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-09-12T14:53:02.701 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-09-12T14:53:02.854 INFO:teuthology.orchestra.run.smithi187.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-09-12T14:53:02.856 INFO:teuthology.orchestra.run.smithi187.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-09-12T14:53:02.856 INFO:teuthology.orchestra.run.smithi187.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-09-12T14:53:02.859 INFO:teuthology.orchestra.run.smithi187.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-09-12T14:53:02.859 INFO:teuthology.orchestra.run.smithi187.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-09-12T14:53:02.863 INFO:teuthology.orchestra.run.smithi187.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-09-12T14:53:02.865 INFO:teuthology.orchestra.run.smithi187.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-09-12T14:53:02.872 INFO:teuthology.orchestra.run.smithi187.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-09-12T14:53:02.873 INFO:teuthology.orchestra.run.smithi187.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.10 [2,372 kB] 2023-09-12T14:53:02.893 INFO:teuthology.orchestra.run.smithi187.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-09-12T14:53:02.894 INFO:teuthology.orchestra.run.smithi187.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-09-12T14:53:02.904 INFO:teuthology.orchestra.run.smithi187.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-09-12T14:53:02.923 INFO:teuthology.orchestra.run.smithi187.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-09-12T14:53:02.926 INFO:teuthology.orchestra.run.smithi187.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-09-12T14:53:02.929 INFO:teuthology.orchestra.run.smithi187.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-09-12T14:53:02.929 INFO:teuthology.orchestra.run.smithi187.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-09-12T14:53:02.930 INFO:teuthology.orchestra.run.smithi187.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-09-12T14:53:02.938 INFO:teuthology.orchestra.run.smithi187.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-09-12T14:53:02.940 INFO:teuthology.orchestra.run.smithi187.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-09-12T14:53:03.063 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-09-12T14:53:03.138 INFO:teuthology.orchestra.run.smithi187.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-09-12T14:53:03.138 INFO:teuthology.orchestra.run.smithi187.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-09-12T14:53:03.138 INFO:teuthology.orchestra.run.smithi187.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-09-12T14:53:03.139 INFO:teuthology.orchestra.run.smithi187.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-09-12T14:53:03.139 INFO:teuthology.orchestra.run.smithi187.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-09-12T14:53:03.139 INFO:teuthology.orchestra.run.smithi187.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-09-12T14:53:03.150 INFO:teuthology.orchestra.run.smithi187.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-09-12T14:53:03.150 INFO:teuthology.orchestra.run.smithi187.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-09-12T14:53:03.503 INFO:teuthology.orchestra.run.smithi187.stdout:Preconfiguring packages ... 2023-09-12T14:53:03.599 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up cephfs-mirror (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:53:03.649 INFO:teuthology.orchestra.run.smithi187.stdout:Fetched 54.6 MB in 1s (61.3 MB/s) 2023-09-12T14:53:03.789 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-09-12T14:53:03.790 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-09-12T14:53:03.790 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package flex. 2023-09-12T14:53:03.883 INFO:teuthology.orchestra.run.smithi187.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 172986 files and directories currently installed.) 2023-09-12T14:53:03.888 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-09-12T14:53:03.930 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking flex (2.6.4-8build2) ... 2023-09-12T14:53:04.333 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-test (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:53:04.460 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up radosgw (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:53:04.497 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.10_amd64.deb ... 2023-09-12T14:53:04.654 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.10) over (3.0.2-0ubuntu1.7) ... 2023-09-12T14:53:04.721 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-09-12T14:53:04.722 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-09-12T14:53:05.054 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.10) ... 2023-09-12T14:53:05.495 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-base (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:53:05.553 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package bison. 2023-09-12T14:53:05.657 INFO:teuthology.orchestra.run.smithi187.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173059 files and directories currently installed.) 2023-09-12T14:53:05.662 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-09-12T14:53:05.698 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-09-12T14:53:05.873 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-09-12T14:53:06.082 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-09-12T14:53:06.110 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-09-12T14:53:06.140 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-09-12T14:53:06.408 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-mds (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:53:06.427 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package dump. 2023-09-12T14:53:06.454 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-09-12T14:53:06.493 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking dump (0.4b47-3) ... 2023-09-12T14:53:06.634 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-09-12T14:53:06.634 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-09-12T14:53:06.772 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package indent. 2023-09-12T14:53:06.799 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../03-indent_2.2.12-1build2_amd64.deb ... 2023-09-12T14:53:06.838 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking indent (2.2.12-1build2) ... 2023-09-12T14:53:07.099 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-09-12T14:53:07.127 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-09-12T14:53:07.166 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-09-12T14:53:07.168 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-mgr (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:53:07.414 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-09-12T14:53:07.414 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-09-12T14:53:07.486 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-09-12T14:53:07.513 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-09-12T14:53:07.544 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-09-12T14:53:07.797 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-09-12T14:53:07.813 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-09-12T14:53:07.857 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-09-12T14:53:07.944 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-osd (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:53:08.183 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libfl2:amd64. 2023-09-12T14:53:08.211 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-09-12T14:53:08.247 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-09-12T14:53:08.248 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-09-12T14:53:08.249 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-09-12T14:53:08.502 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-09-12T14:53:08.518 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-09-12T14:53:08.560 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-09-12T14:53:08.762 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-mgr-k8sevents (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:53:08.805 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libjson-perl. 2023-09-12T14:53:08.832 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-09-12T14:53:08.871 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-09-12T14:53:08.872 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:53:09.023 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-mon (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:53:09.235 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-09-12T14:53:09.235 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-09-12T14:53:09.554 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libllvm14:amd64. 2023-09-12T14:53:09.582 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-09-12T14:53:09.740 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-mgr-cephadm (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:53:09.792 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-09-12T14:53:09.892 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:53:10.017 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-mgr-dashboard (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:53:10.168 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-volume (18.0.0-6088-g2110e007-1jammy) ... 2023-09-12T14:53:10.296 INFO:teuthology.orchestra.run.smithi131.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-09-12T14:53:11.020 INFO:teuthology.orchestra.run.smithi131.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-09-12T14:53:11.473 INFO:teuthology.orchestra.run.smithi131.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-09-12T14:53:11.475 DEBUG:teuthology.orchestra.run.smithi131:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-09-12T14:53:11.503 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-09-12T14:53:11.530 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-09-12T14:53:11.570 INFO:teuthology.orchestra.run.smithi131.stdout:Reading package lists... 2023-09-12T14:53:11.578 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-09-12T14:53:11.752 INFO:teuthology.orchestra.run.smithi131.stdout:Building dependency tree... 2023-09-12T14:53:11.753 INFO:teuthology.orchestra.run.smithi131.stdout:Reading state information... 2023-09-12T14:53:11.788 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libmbim-proxy. 2023-09-12T14:53:11.805 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-09-12T14:53:11.839 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-09-12T14:53:11.915 INFO:teuthology.orchestra.run.smithi131.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-09-12T14:53:11.916 INFO:teuthology.orchestra.run.smithi131.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-09-12T14:53:11.916 INFO:teuthology.orchestra.run.smithi131.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-09-12T14:53:11.916 INFO:teuthology.orchestra.run.smithi131.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-09-12T14:53:11.916 INFO:teuthology.orchestra.run.smithi131.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-09-12T14:53:11.917 INFO:teuthology.orchestra.run.smithi131.stdout:The following packages were automatically installed and are no longer required: 2023-09-12T14:53:11.917 INFO:teuthology.orchestra.run.smithi131.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-09-12T14:53:11.917 INFO:teuthology.orchestra.run.smithi131.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-09-12T14:53:11.917 INFO:teuthology.orchestra.run.smithi131.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-09-12T14:53:11.917 INFO:teuthology.orchestra.run.smithi131.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-09-12T14:53:11.918 INFO:teuthology.orchestra.run.smithi131.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-09-12T14:53:11.919 INFO:teuthology.orchestra.run.smithi131.stdout:Use 'sudo apt autoremove' to remove them. 2023-09-12T14:53:11.919 INFO:teuthology.orchestra.run.smithi131.stdout:The following additional packages will be installed: 2023-09-12T14:53:11.919 INFO:teuthology.orchestra.run.smithi131.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-09-12T14:53:11.920 INFO:teuthology.orchestra.run.smithi131.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-09-12T14:53:11.920 INFO:teuthology.orchestra.run.smithi131.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-09-12T14:53:11.920 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-09-12T14:53:11.920 INFO:teuthology.orchestra.run.smithi131.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-09-12T14:53:11.920 INFO:teuthology.orchestra.run.smithi131.stdout:Suggested packages: 2023-09-12T14:53:11.921 INFO:teuthology.orchestra.run.smithi131.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-09-12T14:53:11.921 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-09-12T14:53:11.921 INFO:teuthology.orchestra.run.smithi131.stdout: libengine-pkcs11-openssl 2023-09-12T14:53:11.954 INFO:teuthology.orchestra.run.smithi131.stdout:The following NEW packages will be installed: 2023-09-12T14:53:11.955 INFO:teuthology.orchestra.run.smithi131.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-09-12T14:53:11.955 INFO:teuthology.orchestra.run.smithi131.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-09-12T14:53:11.956 INFO:teuthology.orchestra.run.smithi131.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-09-12T14:53:11.956 INFO:teuthology.orchestra.run.smithi131.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-09-12T14:53:11.956 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-09-12T14:53:11.957 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-09-12T14:53:11.957 INFO:teuthology.orchestra.run.smithi131.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-09-12T14:53:11.957 INFO:teuthology.orchestra.run.smithi131.stdout:The following packages will be upgraded: 2023-09-12T14:53:11.958 INFO:teuthology.orchestra.run.smithi131.stdout: libssl3 2023-09-12T14:53:12.007 INFO:teuthology.orchestra.run.smithi131.stdout:1 upgraded, 39 newly installed, 0 to remove and 240 not upgraded. 2023-09-12T14:53:12.007 INFO:teuthology.orchestra.run.smithi131.stdout:Need to get 54.6 MB of archives. 2023-09-12T14:53:12.008 INFO:teuthology.orchestra.run.smithi131.stdout:After this operation, 202 MB of additional disk space will be used. 2023-09-12T14:53:12.008 INFO:teuthology.orchestra.run.smithi131.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-09-12T14:53:12.105 INFO:teuthology.orchestra.run.smithi131.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.10 [1,901 kB] 2023-09-12T14:53:12.158 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-09-12T14:53:12.165 INFO:teuthology.orchestra.run.smithi131.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-09-12T14:53:12.179 INFO:teuthology.orchestra.run.smithi131.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-09-12T14:53:12.180 INFO:teuthology.orchestra.run.smithi131.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-09-12T14:53:12.182 INFO:teuthology.orchestra.run.smithi131.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 indent amd64 2.2.12-1build2 [90.8 kB] 2023-09-12T14:53:12.184 INFO:teuthology.orchestra.run.smithi131.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-09-12T14:53:12.185 INFO:teuthology.orchestra.run.smithi131.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-09-12T14:53:12.186 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-09-12T14:53:12.187 INFO:teuthology.orchestra.run.smithi131.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-09-12T14:53:12.188 INFO:teuthology.orchestra.run.smithi131.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-09-12T14:53:12.189 INFO:teuthology.orchestra.run.smithi131.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-09-12T14:53:12.189 INFO:teuthology.orchestra.run.smithi131.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-09-12T14:53:12.202 INFO:teuthology.orchestra.run.smithi131.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-09-12T14:53:12.224 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-09-12T14:53:12.519 INFO:teuthology.orchestra.run.smithi131.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-09-12T14:53:12.521 INFO:teuthology.orchestra.run.smithi131.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-09-12T14:53:12.521 INFO:teuthology.orchestra.run.smithi131.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-09-12T14:53:12.523 INFO:teuthology.orchestra.run.smithi131.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-09-12T14:53:12.524 INFO:teuthology.orchestra.run.smithi131.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-09-12T14:53:12.528 INFO:teuthology.orchestra.run.smithi131.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-09-12T14:53:12.529 INFO:teuthology.orchestra.run.smithi131.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-09-12T14:53:12.535 INFO:teuthology.orchestra.run.smithi131.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-09-12T14:53:12.536 INFO:teuthology.orchestra.run.smithi131.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.10 [2,372 kB] 2023-09-12T14:53:12.555 INFO:teuthology.orchestra.run.smithi131.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-09-12T14:53:12.556 INFO:teuthology.orchestra.run.smithi131.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-09-12T14:53:12.564 INFO:teuthology.orchestra.run.smithi131.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-09-12T14:53:12.585 INFO:teuthology.orchestra.run.smithi131.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-09-12T14:53:12.585 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libndp0:amd64. 2023-09-12T14:53:12.587 INFO:teuthology.orchestra.run.smithi131.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-09-12T14:53:12.588 INFO:teuthology.orchestra.run.smithi131.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-09-12T14:53:12.588 INFO:teuthology.orchestra.run.smithi131.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-09-12T14:53:12.589 INFO:teuthology.orchestra.run.smithi131.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-09-12T14:53:12.599 INFO:teuthology.orchestra.run.smithi131.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-09-12T14:53:12.600 INFO:teuthology.orchestra.run.smithi131.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-09-12T14:53:12.613 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-09-12T14:53:12.652 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-09-12T14:53:12.801 INFO:teuthology.orchestra.run.smithi131.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-09-12T14:53:12.801 INFO:teuthology.orchestra.run.smithi131.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-09-12T14:53:12.802 INFO:teuthology.orchestra.run.smithi131.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-09-12T14:53:12.802 INFO:teuthology.orchestra.run.smithi131.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-09-12T14:53:12.802 INFO:teuthology.orchestra.run.smithi131.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-09-12T14:53:12.803 INFO:teuthology.orchestra.run.smithi131.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-09-12T14:53:12.813 INFO:teuthology.orchestra.run.smithi131.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-09-12T14:53:12.814 INFO:teuthology.orchestra.run.smithi131.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-09-12T14:53:12.938 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libnm0:amd64. 2023-09-12T14:53:12.966 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-09-12T14:53:13.005 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-09-12T14:53:13.211 INFO:teuthology.orchestra.run.smithi131.stdout:Preconfiguring packages ... 2023-09-12T14:53:13.358 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libpq5:amd64. 2023-09-12T14:53:13.361 INFO:teuthology.orchestra.run.smithi131.stdout:Fetched 54.6 MB in 1s (64.2 MB/s) 2023-09-12T14:53:13.385 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-09-12T14:53:13.424 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-09-12T14:53:13.502 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package flex. 2023-09-12T14:53:13.590 INFO:teuthology.orchestra.run.smithi131.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 172986 files and directories currently installed.) 2023-09-12T14:53:13.595 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-09-12T14:53:13.685 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking flex (2.6.4-8build2) ... 2023-09-12T14:53:13.760 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-09-12T14:53:13.788 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-09-12T14:53:13.826 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-09-12T14:53:14.121 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libqmi-proxy. 2023-09-12T14:53:14.137 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-09-12T14:53:14.179 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-09-12T14:53:14.366 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.10_amd64.deb ... 2023-09-12T14:53:14.406 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-09-12T14:53:14.422 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.10_amd64.deb ... 2023-09-12T14:53:14.456 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.10) ... 2023-09-12T14:53:14.539 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.10) over (3.0.2-0ubuntu1.7) ... 2023-09-12T14:53:14.957 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.10) ... 2023-09-12T14:53:15.019 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-09-12T14:53:15.047 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-09-12T14:53:15.085 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-09-12T14:53:15.438 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package modemmanager. 2023-09-12T14:53:15.439 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package bison. 2023-09-12T14:53:15.466 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-09-12T14:53:15.496 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-09-12T14:53:15.502 INFO:teuthology.orchestra.run.smithi131.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173059 files and directories currently installed.) 2023-09-12T14:53:15.508 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-09-12T14:53:15.559 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-09-12T14:53:15.993 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-09-12T14:53:16.021 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-09-12T14:53:16.061 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-09-12T14:53:16.145 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package network-manager. 2023-09-12T14:53:16.172 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-09-12T14:53:16.219 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-09-12T14:53:16.404 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package dump. 2023-09-12T14:53:16.432 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-09-12T14:53:16.479 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking dump (0.4b47-3) ... 2023-09-12T14:53:16.767 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ppp. 2023-09-12T14:53:16.774 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package indent. 2023-09-12T14:53:16.790 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../03-indent_2.2.12-1build2_amd64.deb ... 2023-09-12T14:53:16.795 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-09-12T14:53:16.832 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking indent (2.2.12-1build2) ... 2023-09-12T14:53:17.093 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-09-12T14:53:17.109 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-09-12T14:53:17.111 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-09-12T14:53:17.151 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-09-12T14:53:17.399 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package pptp-linux. 2023-09-12T14:53:17.415 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-09-12T14:53:17.449 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-09-12T14:53:17.497 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-09-12T14:53:17.524 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-09-12T14:53:17.571 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-09-12T14:53:17.701 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package network-manager-pptp. 2023-09-12T14:53:17.717 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-09-12T14:53:17.760 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-09-12T14:53:17.874 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-09-12T14:53:17.902 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-09-12T14:53:17.949 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-09-12T14:53:18.046 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package postgresql-client-common. 2023-09-12T14:53:18.073 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-09-12T14:53:18.113 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking postgresql-client-common (238) ... 2023-09-12T14:53:18.269 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libfl2:amd64. 2023-09-12T14:53:18.296 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-09-12T14:53:18.335 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-09-12T14:53:18.425 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package postgresql-client-14. 2023-09-12T14:53:18.452 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-09-12T14:53:18.491 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-09-12T14:53:18.596 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-09-12T14:53:18.612 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-09-12T14:53:18.654 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-09-12T14:53:19.007 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libjson-perl. 2023-09-12T14:53:19.035 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-09-12T14:53:19.071 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package postgresql-common. 2023-09-12T14:53:19.082 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-09-12T14:53:19.099 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-09-12T14:53:19.156 INFO:teuthology.orchestra.run.smithi187.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-09-12T14:53:19.185 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking postgresql-common (238) ... 2023-09-12T14:53:19.426 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libllvm14:amd64. 2023-09-12T14:53:19.454 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-09-12T14:53:19.493 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-09-12T14:53:19.678 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package postgresql-14. 2023-09-12T14:53:19.706 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-09-12T14:53:19.748 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-09-12T14:53:21.131 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package postgresql. 2023-09-12T14:53:21.148 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-09-12T14:53:21.187 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-09-12T14:53:21.190 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking postgresql (14+238) ... 2023-09-12T14:53:21.214 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-09-12T14:53:21.254 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-09-12T14:53:21.433 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package postgresql-client. 2023-09-12T14:53:21.450 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-09-12T14:53:21.492 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking postgresql-client (14+238) ... 2023-09-12T14:53:21.531 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libmbim-proxy. 2023-09-12T14:53:21.559 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-09-12T14:53:21.598 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-09-12T14:53:21.736 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package postgresql-contrib. 2023-09-12T14:53:21.764 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-09-12T14:53:21.802 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking postgresql-contrib (14+238) ... 2023-09-12T14:53:22.000 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-09-12T14:53:22.028 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-09-12T14:53:22.075 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-09-12T14:53:22.121 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package tcl8.6. 2023-09-12T14:53:22.149 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-09-12T14:53:22.188 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-09-12T14:53:22.436 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libndp0:amd64. 2023-09-12T14:53:22.440 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package tcl. 2023-09-12T14:53:22.457 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-09-12T14:53:22.464 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-09-12T14:53:22.503 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-09-12T14:53:22.503 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-09-12T14:53:22.852 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package wpasupplicant. 2023-09-12T14:53:22.865 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libnm0:amd64. 2023-09-12T14:53:22.880 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-09-12T14:53:22.892 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-09-12T14:53:22.918 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-09-12T14:53:22.930 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-09-12T14:53:23.273 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-09-12T14:53:23.301 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-09-12T14:53:23.325 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libpq5:amd64. 2023-09-12T14:53:23.339 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-09-12T14:53:23.353 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-09-12T14:53:23.400 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-09-12T14:53:23.659 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package usb-modeswitch. 2023-09-12T14:53:23.687 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-09-12T14:53:23.725 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-09-12T14:53:23.778 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-09-12T14:53:23.805 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-09-12T14:53:23.844 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-09-12T14:53:24.011 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up postgresql-client-common (238) ... 2023-09-12T14:53:24.172 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libqmi-proxy. 2023-09-12T14:53:24.199 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-09-12T14:53:24.232 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up flex (2.6.4-8build2) ... 2023-09-12T14:53:24.238 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-09-12T14:53:24.358 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-09-12T14:53:24.449 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-09-12T14:53:24.465 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.10_amd64.deb ... 2023-09-12T14:53:24.483 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-09-12T14:53:24.498 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.10) ... 2023-09-12T14:53:24.609 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-09-12T14:53:24.761 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-09-12T14:53:24.870 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-09-12T14:53:25.052 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-09-12T14:53:25.080 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-09-12T14:53:25.119 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-09-12T14:53:25.342 INFO:teuthology.orchestra.run.smithi187.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-09-12T14:53:25.420 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-09-12T14:53:25.480 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package modemmanager. 2023-09-12T14:53:25.507 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-09-12T14:53:25.546 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-09-12T14:53:25.748 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-09-12T14:53:25.748 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-09-12T14:53:26.211 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package network-manager. 2023-09-12T14:53:26.239 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-09-12T14:53:26.272 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-09-12T14:53:26.285 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-09-12T14:53:26.398 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.10) ... 2023-09-12T14:53:26.524 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-09-12T14:53:26.650 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-09-12T14:53:26.776 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-09-12T14:53:26.902 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-09-12T14:53:26.926 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ppp. 2023-09-12T14:53:26.954 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-09-12T14:53:27.003 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-09-12T14:53:27.268 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-09-12T14:53:27.592 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-09-12T14:53:27.605 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package pptp-linux. 2023-09-12T14:53:27.633 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-09-12T14:53:27.672 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-09-12T14:53:27.718 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up dump (0.4b47-3) ... 2023-09-12T14:53:27.811 INFO:teuthology.orchestra.run.smithi187.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-09-12T14:53:27.877 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-09-12T14:53:27.916 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package network-manager-pptp. 2023-09-12T14:53:27.932 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-09-12T14:53:27.974 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-09-12T14:53:27.978 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-09-12T14:53:28.121 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up tcl (8.6.11+1build2) ... 2023-09-12T14:53:28.210 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package postgresql-client-common. 2023-09-12T14:53:28.222 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libjson-perl (4.04000-1) ... 2023-09-12T14:53:28.226 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-09-12T14:53:28.269 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking postgresql-client-common (238) ... 2023-09-12T14:53:28.348 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-09-12T14:53:28.439 INFO:teuthology.orchestra.run.smithi187.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-09-12T14:53:28.508 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up indent (2.2.12-1build2) ... 2023-09-12T14:53:28.555 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package postgresql-client-14. 2023-09-12T14:53:28.585 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-09-12T14:53:28.622 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-09-12T14:53:28.659 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-09-12T14:53:28.835 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-09-12T14:53:28.961 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-09-12T14:53:29.062 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-09-12T14:53:29.197 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up postgresql-client (14+238) ... 2023-09-12T14:53:29.306 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-09-12T14:53:29.386 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package postgresql-common. 2023-09-12T14:53:29.413 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-09-12T14:53:29.470 INFO:teuthology.orchestra.run.smithi131.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-09-12T14:53:29.500 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking postgresql-common (238) ... 2023-09-12T14:53:29.812 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:29.812 INFO:teuthology.orchestra.run.smithi187.stdout:The following network interfaces were found in /etc/network/interfaces 2023-09-12T14:53:29.812 INFO:teuthology.orchestra.run.smithi187.stdout:which means they are currently configured by ifupdown: 2023-09-12T14:53:29.812 INFO:teuthology.orchestra.run.smithi187.stdout:- enp3s0f1 2023-09-12T14:53:29.812 INFO:teuthology.orchestra.run.smithi187.stdout:If you want to manage those interfaces with NetworkManager instead 2023-09-12T14:53:29.813 INFO:teuthology.orchestra.run.smithi187.stdout:remove their configuration from /etc/network/interfaces. 2023-09-12T14:53:29.813 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:29.892 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package postgresql-14. 2023-09-12T14:53:29.920 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-09-12T14:53:29.938 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-09-12T14:53:29.962 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-09-12T14:53:30.298 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-09-12T14:53:30.604 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-09-12T14:53:31.109 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-09-12T14:53:31.162 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package postgresql. 2023-09-12T14:53:31.178 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-09-12T14:53:31.220 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking postgresql (14+238) ... 2023-09-12T14:53:31.252 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-09-12T14:53:31.353 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-09-12T14:53:31.454 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up postgresql-common (238) ... 2023-09-12T14:53:31.823 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package postgresql-client. 2023-09-12T14:53:31.839 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-09-12T14:53:31.872 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking postgresql-client (14+238) ... 2023-09-12T14:53:32.083 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package postgresql-contrib. 2023-09-12T14:53:32.099 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-09-12T14:53:32.133 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking postgresql-contrib (14+238) ... 2023-09-12T14:53:32.305 INFO:teuthology.orchestra.run.smithi187.stdout:Adding user postgres to group ssl-cert 2023-09-12T14:53:32.452 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package tcl8.6. 2023-09-12T14:53:32.480 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-09-12T14:53:32.518 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-09-12T14:53:32.557 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:32.558 INFO:teuthology.orchestra.run.smithi187.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-09-12T14:53:32.769 INFO:teuthology.orchestra.run.smithi187.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-09-12T14:53:32.769 INFO:teuthology.orchestra.run.smithi187.stdout:Removing obsolete dictionary files: 2023-09-12T14:53:32.829 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package tcl. 2023-09-12T14:53:32.845 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-09-12T14:53:32.892 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-09-12T14:53:33.182 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package wpasupplicant. 2023-09-12T14:53:33.198 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-09-12T14:53:33.232 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-09-12T14:53:33.335 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-09-12T14:53:33.612 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-09-12T14:53:33.640 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-09-12T14:53:33.678 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-09-12T14:53:33.722 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-09-12T14:53:34.057 INFO:teuthology.orchestra.run.smithi187.stdout:Creating new PostgreSQL cluster 14/main ... 2023-09-12T14:53:34.065 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package usb-modeswitch. 2023-09-12T14:53:34.082 INFO:teuthology.orchestra.run.smithi187.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-09-12T14:53:34.093 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-09-12T14:53:34.135 INFO:teuthology.orchestra.run.smithi187.stdout:The files belonging to this database system will be owned by user "postgres". 2023-09-12T14:53:34.135 INFO:teuthology.orchestra.run.smithi187.stdout:This user must also own the server process. 2023-09-12T14:53:34.136 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:34.136 INFO:teuthology.orchestra.run.smithi187.stdout:The database cluster will be initialized with locale "en_US". 2023-09-12T14:53:34.136 INFO:teuthology.orchestra.run.smithi187.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-09-12T14:53:34.136 INFO:teuthology.orchestra.run.smithi187.stdout:The default text search configuration will be set to "english". 2023-09-12T14:53:34.136 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:34.137 INFO:teuthology.orchestra.run.smithi187.stdout:Data page checksums are disabled. 2023-09-12T14:53:34.137 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:34.137 INFO:teuthology.orchestra.run.smithi187.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-09-12T14:53:34.137 INFO:teuthology.orchestra.run.smithi187.stdout:creating subdirectories ... ok 2023-09-12T14:53:34.137 INFO:teuthology.orchestra.run.smithi187.stdout:selecting dynamic shared memory implementation ... posix 2023-09-12T14:53:34.153 INFO:teuthology.orchestra.run.smithi187.stdout:selecting default max_connections ... 100 2023-09-12T14:53:34.156 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-09-12T14:53:34.173 INFO:teuthology.orchestra.run.smithi187.stdout:selecting default shared_buffers ... 128MB 2023-09-12T14:53:34.189 INFO:teuthology.orchestra.run.smithi187.stdout:selecting default time zone ... Etc/UTC 2023-09-12T14:53:34.190 INFO:teuthology.orchestra.run.smithi187.stdout:creating configuration files ... ok 2023-09-12T14:53:34.307 INFO:teuthology.orchestra.run.smithi187.stdout:running bootstrap script ... ok 2023-09-12T14:53:34.433 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up postgresql-client-common (238) ... 2023-09-12T14:53:34.645 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up flex (2.6.4-8build2) ... 2023-09-12T14:53:34.746 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-09-12T14:53:34.872 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-09-12T14:53:34.951 INFO:teuthology.orchestra.run.smithi187.stdout:performing post-bootstrap initialization ... ok 2023-09-12T14:53:34.998 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-09-12T14:53:35.124 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-09-12T14:53:35.250 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-09-12T14:53:35.583 INFO:teuthology.orchestra.run.smithi187.stdout:syncing data to disk ... ok 2023-09-12T14:53:35.740 INFO:teuthology.orchestra.run.smithi131.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-09-12T14:53:35.822 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-09-12T14:53:35.983 INFO:teuthology.orchestra.run.smithi187.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-09-12T14:53:36.255 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-09-12T14:53:36.255 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-09-12T14:53:36.792 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-09-12T14:53:36.918 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.10) ... 2023-09-12T14:53:37.052 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-09-12T14:53:37.178 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-09-12T14:53:37.304 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-09-12T14:53:37.430 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-09-12T14:53:37.565 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-09-12T14:53:38.295 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-09-12T14:53:38.421 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up dump (0.4b47-3) ... 2023-09-12T14:53:38.523 INFO:teuthology.orchestra.run.smithi131.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-09-12T14:53:38.589 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-09-12T14:53:38.723 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-09-12T14:53:38.770 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-09-12T14:53:38.874 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up tcl (8.6.11+1build2) ... 2023-09-12T14:53:38.879 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up postgresql-contrib (14+238) ... 2023-09-12T14:53:39.014 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up postgresql (14+238) ... 2023-09-12T14:53:39.025 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libjson-perl (4.04000-1) ... 2023-09-12T14:53:39.140 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-09-12T14:53:39.168 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-09-12T14:53:39.266 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-09-12T14:53:39.275 INFO:teuthology.orchestra.run.smithi131.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-09-12T14:53:39.336 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up indent (2.2.12-1build2) ... 2023-09-12T14:53:39.462 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-09-12T14:53:39.522 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-09-12T14:53:39.523 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-09-12T14:53:39.639 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-09-12T14:53:39.765 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-09-12T14:53:39.908 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-09-12T14:53:40.109 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up postgresql-client (14+238) ... 2023-09-12T14:53:40.160 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-09-12T14:53:40.260 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-09-12T14:53:40.473 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-09-12T14:53:40.622 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-09-12T14:53:40.754 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:40.755 INFO:teuthology.orchestra.run.smithi131.stdout:The following network interfaces were found in /etc/network/interfaces 2023-09-12T14:53:40.755 INFO:teuthology.orchestra.run.smithi131.stdout:which means they are currently configured by ifupdown: 2023-09-12T14:53:40.756 INFO:teuthology.orchestra.run.smithi131.stdout:- enp3s0f1 2023-09-12T14:53:40.756 INFO:teuthology.orchestra.run.smithi131.stdout:If you want to manage those interfaces with NetworkManager instead 2023-09-12T14:53:40.756 INFO:teuthology.orchestra.run.smithi131.stdout:remove their configuration from /etc/network/interfaces. 2023-09-12T14:53:40.757 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:40.871 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-09-12T14:53:41.212 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-09-12T14:53:41.572 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-09-12T14:53:41.909 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-09-12T14:53:42.016 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-09-12T14:53:42.162 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-09-12T14:53:42.297 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-09-12T14:53:42.423 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-09-12T14:53:42.591 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up postgresql-common (238) ... 2023-09-12T14:53:42.641 INFO:teuthology.orchestra.run.smithi187.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-09-12T14:53:42.643 DEBUG:teuthology.parallel:result is None 2023-09-12T14:53:43.518 INFO:teuthology.orchestra.run.smithi131.stdout:Adding user postgres to group ssl-cert 2023-09-12T14:53:43.752 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:43.752 INFO:teuthology.orchestra.run.smithi131.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-09-12T14:53:43.936 INFO:teuthology.orchestra.run.smithi131.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-09-12T14:53:43.936 INFO:teuthology.orchestra.run.smithi131.stdout:Removing obsolete dictionary files: 2023-09-12T14:53:44.551 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-09-12T14:53:44.985 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-09-12T14:53:45.319 INFO:teuthology.orchestra.run.smithi131.stdout:Creating new PostgreSQL cluster 14/main ... 2023-09-12T14:53:45.342 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-09-12T14:53:45.384 INFO:teuthology.orchestra.run.smithi131.stdout:The files belonging to this database system will be owned by user "postgres". 2023-09-12T14:53:45.385 INFO:teuthology.orchestra.run.smithi131.stdout:This user must also own the server process. 2023-09-12T14:53:45.385 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:45.385 INFO:teuthology.orchestra.run.smithi131.stdout:The database cluster will be initialized with locale "en_US". 2023-09-12T14:53:45.385 INFO:teuthology.orchestra.run.smithi131.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-09-12T14:53:45.385 INFO:teuthology.orchestra.run.smithi131.stdout:The default text search configuration will be set to "english". 2023-09-12T14:53:45.385 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:45.386 INFO:teuthology.orchestra.run.smithi131.stdout:Data page checksums are disabled. 2023-09-12T14:53:45.386 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:45.386 INFO:teuthology.orchestra.run.smithi131.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-09-12T14:53:45.386 INFO:teuthology.orchestra.run.smithi131.stdout:creating subdirectories ... ok 2023-09-12T14:53:45.386 INFO:teuthology.orchestra.run.smithi131.stdout:selecting dynamic shared memory implementation ... posix 2023-09-12T14:53:45.401 INFO:teuthology.orchestra.run.smithi131.stdout:selecting default max_connections ... 100 2023-09-12T14:53:45.422 INFO:teuthology.orchestra.run.smithi131.stdout:selecting default shared_buffers ... 128MB 2023-09-12T14:53:45.432 INFO:teuthology.orchestra.run.smithi131.stdout:selecting default time zone ... Etc/UTC 2023-09-12T14:53:45.433 INFO:teuthology.orchestra.run.smithi131.stdout:creating configuration files ... ok 2023-09-12T14:53:45.546 INFO:teuthology.orchestra.run.smithi131.stdout:running bootstrap script ... ok 2023-09-12T14:53:46.166 INFO:teuthology.orchestra.run.smithi131.stdout:performing post-bootstrap initialization ... ok 2023-09-12T14:53:46.762 INFO:teuthology.orchestra.run.smithi131.stdout:syncing data to disk ... ok 2023-09-12T14:53:47.210 INFO:teuthology.orchestra.run.smithi131.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-09-12T14:53:50.009 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-09-12T14:53:50.135 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up postgresql-contrib (14+238) ... 2023-09-12T14:53:50.261 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up postgresql (14+238) ... 2023-09-12T14:53:50.387 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-09-12T14:53:50.513 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-09-12T14:53:50.767 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-09-12T14:53:50.767 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-09-12T14:53:51.366 INFO:teuthology.orchestra.run.smithi131.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-09-12T14:53:51.660 INFO:teuthology.orchestra.run.smithi131.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-09-12T14:53:51.803 INFO:teuthology.orchestra.run.smithi131.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-09-12T14:53:53.165 INFO:teuthology.orchestra.run.smithi131.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-09-12T14:53:53.265 INFO:teuthology.orchestra.run.smithi131.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-09-12T14:53:53.878 INFO:teuthology.orchestra.run.smithi131.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-09-12T14:53:53.880 DEBUG:teuthology.parallel:result is None 2023-09-12T14:53:53.881 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2110e00747b31a0d73768c9e5229da18b26b8aa0 2023-09-12T14:53:54.098 DEBUG:teuthology.orchestra.run.smithi131:> dpkg-query -W -f '${Version}' ceph 2023-09-12T14:53:54.117 INFO:teuthology.orchestra.run.smithi131.stdout:18.0.0-6088-g2110e007-1jammy 2023-09-12T14:53:54.117 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6088-g2110e007-1jammy 2023-09-12T14:53:54.118 INFO:teuthology.task.install:The correct ceph version 18.0.0-6088-g2110e007-1jammy is installed. 2023-09-12T14:53:54.118 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2110e00747b31a0d73768c9e5229da18b26b8aa0 2023-09-12T14:53:54.332 DEBUG:teuthology.orchestra.run.smithi187:> dpkg-query -W -f '${Version}' ceph 2023-09-12T14:53:54.352 INFO:teuthology.orchestra.run.smithi187.stdout:18.0.0-6088-g2110e007-1jammy 2023-09-12T14:53:54.353 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6088-g2110e007-1jammy 2023-09-12T14:53:54.353 INFO:teuthology.task.install:The correct ceph version 18.0.0-6088-g2110e007-1jammy is installed. 2023-09-12T14:53:54.353 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-09-12T14:53:54.354 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:53:54.354 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-12T14:53:54.369 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:53:54.369 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-12T14:53:54.409 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-09-12T14:53:54.410 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:53:54.410 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/daemon-helper 2023-09-12T14:53:54.425 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-12T14:53:54.480 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:53:54.481 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/daemon-helper 2023-09-12T14:53:54.495 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-12T14:53:54.547 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-09-12T14:53:54.547 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:53:54.548 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-12T14:53:54.560 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-12T14:53:54.614 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:53:54.615 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-12T14:53:54.630 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-12T14:53:54.683 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-09-12T14:53:54.683 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:53:54.683 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/stdin-killer 2023-09-12T14:53:54.697 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-12T14:53:54.753 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:53:54.753 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/stdin-killer 2023-09-12T14:53:54.766 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-12T14:53:54.821 INFO:teuthology.run_tasks:Running task ceph... 2023-09-12T14:53:54.887 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-09-12T14:53:54.887 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 777 /var/log/ceph 2023-09-12T14:53:54.890 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 777 /var/log/ceph 2023-09-12T14:53:54.903 INFO:tasks.ceph:Disabling ceph logrotate... 2023-09-12T14:53:54.903 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /etc/logrotate.d/ceph 2023-09-12T14:53:54.951 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /etc/logrotate.d/ceph 2023-09-12T14:53:54.963 INFO:tasks.ceph:Creating extra log directories... 2023-09-12T14:53:54.963 DEBUG:teuthology.orchestra.run.smithi131:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-09-12T14:53:55.003 DEBUG:teuthology.orchestra.run.smithi187:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-09-12T14:53:55.024 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-09-12T14:53:55.025 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-09-12T14:53:55.025 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-09-12T14:53:55.025 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:53:55.026 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-09-12T14:53:55.062 DEBUG:teuthology.orchestra.run.smithi131:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-09-12T14:53:55.137 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:53:55.137 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-09-12T14:53:55.143 DEBUG:teuthology.orchestra.run.smithi187:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-09-12T14:53:55.229 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-09-12T14:53:55.230 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-09-12T14:53:55.230 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395049', 'branch': 'wip-batrick-testing-20230912.122437', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7395049', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-batrick-testing-20230912.122437'}, 'ceph': {'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '2110e00747b31a0d73768c9e5229da18b26b8aa0'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '2110e00747b31a0d73768c9e5229da18b26b8aa0'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-batrick-testing-20230912.122437', 'sha1': '3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 9684, 'sha1': '2110e00747b31a0d73768c9e5229da18b26b8aa0', 'sleep_before_teardown': 0, 'subset': '236/256', 'suite': 'fs', 'suite_branch': 'wip-batrick-testing-20230912.122437', 'suite_path': '/home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/batrick/ceph.git', 'suite_sha1': '3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi131.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDimC56/hFFPRUSSnJw57ISHBvV1b41xUIVLsFaU/Q5jjFtRty1ECTHJAz1lZsZYADp2Hs6BIyUbsb23UEm5CDs=', 'smithi187.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPlylNRPAXfYR2xFtC39/4ZS32+rbGLFQP0faK3NmYDP8iUE2PbEIiYiJxboXIcK55ZSOLyk2idPQ2YJ9Br2zsM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2110e00747b31a0d73768c9e5229da18b26b8aa0'}}, {'ceph': {'create_rbd_pool': False, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '2110e00747b31a0d73768c9e5229da18b26b8aa0', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_quota']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-09-12_14:07:50', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.1394'} 2023-09-12T14:53:55.231 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-09-12T14:53:55.234 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-09-12T14:53:55.268 DEBUG:teuthology.orchestra.run.smithi131:> sudo install -d -m0777 -- /var/run/ceph 2023-09-12T14:53:55.283 DEBUG:teuthology.orchestra.run.smithi187:> sudo install -d -m0777 -- /var/run/ceph 2023-09-12T14:53:55.324 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:53:55.324 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/scratch_devs of=/dev/stdout 2023-09-12T14:53:55.342 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-12T14:53:55.342 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_1 2023-09-12T14:53:55.390 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-12T14:53:55.390 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-12T14:53:55.390 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 886 Links: 1 2023-09-12T14:53:55.391 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-12T14:53:55.391 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2023-09-12 14:47:47.962696205 +0000 2023-09-12T14:53:55.391 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2023-09-12 14:47:47.530699585 +0000 2023-09-12T14:53:55.391 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2023-09-12 14:47:47.530699585 +0000 2023-09-12T14:53:55.391 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: - 2023-09-12T14:53:55.392 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-12T14:53:55.444 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2023-09-12T14:53:55.445 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2023-09-12T14:53:55.445 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000388129 s, 1.3 MB/s 2023-09-12T14:53:55.446 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-12T14:53:55.495 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_2 2023-09-12T14:53:55.541 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-12T14:53:55.542 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-12T14:53:55.542 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 897 Links: 1 2023-09-12T14:53:55.542 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-12T14:53:55.542 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2023-09-12 14:47:48.190694421 +0000 2023-09-12T14:53:55.542 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2023-09-12 14:47:48.186694451 +0000 2023-09-12T14:53:55.543 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2023-09-12 14:47:48.186694451 +0000 2023-09-12T14:53:55.543 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: - 2023-09-12T14:53:55.543 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-12T14:53:55.596 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2023-09-12T14:53:55.597 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2023-09-12T14:53:55.597 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.0003773 s, 1.4 MB/s 2023-09-12T14:53:55.598 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-12T14:53:55.647 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_3 2023-09-12T14:53:55.698 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-12T14:53:55.698 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-12T14:53:55.698 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 920 Links: 1 2023-09-12T14:53:55.698 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-12T14:53:55.699 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2023-09-12 14:47:49.158686841 +0000 2023-09-12T14:53:55.699 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2023-09-12 14:47:48.790689723 +0000 2023-09-12T14:53:55.699 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2023-09-12 14:47:48.790689723 +0000 2023-09-12T14:53:55.699 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: - 2023-09-12T14:53:55.699 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-12T14:53:55.752 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2023-09-12T14:53:55.756 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2023-09-12T14:53:55.756 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000348546 s, 1.5 MB/s 2023-09-12T14:53:55.757 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-12T14:53:55.807 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_4 2023-09-12T14:53:55.854 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-12T14:53:55.854 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-12T14:53:55.854 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 936 Links: 1 2023-09-12T14:53:55.855 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-12T14:53:55.855 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2023-09-12 14:47:49.786681920 +0000 2023-09-12T14:53:55.855 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2023-09-12 14:47:49.422684773 +0000 2023-09-12T14:53:55.855 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2023-09-12 14:47:49.422684773 +0000 2023-09-12T14:53:55.855 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: - 2023-09-12T14:53:55.856 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-12T14:53:55.907 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2023-09-12T14:53:55.908 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2023-09-12T14:53:55.908 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000386184 s, 1.3 MB/s 2023-09-12T14:53:55.909 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-12T14:53:55.959 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-12T14:53:55.960 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:53:55.960 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/scratch_devs of=/dev/stdout 2023-09-12T14:53:55.966 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-12T14:53:55.966 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_1 2023-09-12T14:53:56.013 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-12T14:53:56.013 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-12T14:53:56.013 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 874 Links: 1 2023-09-12T14:53:56.013 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-12T14:53:56.014 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2023-09-12 14:47:53.317605744 +0000 2023-09-12T14:53:56.014 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2023-09-12 14:47:52.953608996 +0000 2023-09-12T14:53:56.014 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2023-09-12 14:47:52.953608996 +0000 2023-09-12T14:53:56.014 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: - 2023-09-12T14:53:56.015 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-12T14:53:56.067 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2023-09-12T14:53:56.067 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2023-09-12T14:53:56.067 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000402543 s, 1.3 MB/s 2023-09-12T14:53:56.068 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-12T14:53:56.118 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_2 2023-09-12T14:53:56.168 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-12T14:53:56.168 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-12T14:53:56.169 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 892 Links: 1 2023-09-12T14:53:56.169 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-12T14:53:56.169 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2023-09-12 14:47:53.909600457 +0000 2023-09-12T14:53:56.169 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2023-09-12 14:47:53.569603493 +0000 2023-09-12T14:53:56.169 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2023-09-12 14:47:53.569603493 +0000 2023-09-12T14:53:56.170 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: - 2023-09-12T14:53:56.170 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-12T14:53:56.223 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2023-09-12T14:53:56.223 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2023-09-12T14:53:56.223 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.00040902 s, 1.3 MB/s 2023-09-12T14:53:56.224 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-12T14:53:56.273 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_3 2023-09-12T14:53:56.320 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-12T14:53:56.320 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-12T14:53:56.321 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 909 Links: 1 2023-09-12T14:53:56.321 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-12T14:53:56.321 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2023-09-12 14:47:54.137598421 +0000 2023-09-12T14:53:56.321 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2023-09-12 14:47:54.133598455 +0000 2023-09-12T14:53:56.322 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2023-09-12 14:47:54.133598455 +0000 2023-09-12T14:53:56.322 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: - 2023-09-12T14:53:56.322 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-12T14:53:56.374 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2023-09-12T14:53:56.375 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2023-09-12T14:53:56.375 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000387657 s, 1.3 MB/s 2023-09-12T14:53:56.376 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-12T14:53:56.425 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_4 2023-09-12T14:53:56.472 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-12T14:53:56.472 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-12T14:53:56.472 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 929 Links: 1 2023-09-12T14:53:56.473 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-12T14:53:56.473 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2023-09-12 14:47:55.125589595 +0000 2023-09-12T14:53:56.473 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2023-09-12 14:47:54.761592847 +0000 2023-09-12T14:53:56.473 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2023-09-12 14:47:54.761592847 +0000 2023-09-12T14:53:56.473 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: - 2023-09-12T14:53:56.474 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-12T14:53:56.526 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2023-09-12T14:53:56.527 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2023-09-12T14:53:56.527 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000384367 s, 1.3 MB/s 2023-09-12T14:53:56.528 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-12T14:53:56.577 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-12T14:53:56.577 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi131.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi187.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-09-12T14:53:56.578 INFO:tasks.ceph:Generating config... 2023-09-12T14:53:56.579 INFO:tasks.ceph:[client] pid file = 2023-09-12T14:53:56.579 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-09-12T14:53:56.580 INFO:tasks.ceph:[mgr] debug ms = 1 2023-09-12T14:53:56.580 INFO:tasks.ceph:[mon] debug mon = 20 2023-09-12T14:53:56.580 INFO:tasks.ceph:[mon] debug ms = 1 2023-09-12T14:53:56.580 INFO:tasks.ceph:[mon] debug paxos = 20 2023-09-12T14:53:56.581 INFO:tasks.ceph:[osd] bdev async discard = True 2023-09-12T14:53:56.581 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-09-12T14:53:56.581 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-09-12T14:53:56.581 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-09-12T14:53:56.582 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-09-12T14:53:56.582 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-09-12T14:53:56.582 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-09-12T14:53:56.583 INFO:tasks.ceph:[osd] debug ms = 1 2023-09-12T14:53:56.583 INFO:tasks.ceph:[osd] debug osd = 20 2023-09-12T14:53:56.583 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-09-12T14:53:56.583 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-09-12T14:53:56.584 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-09-12T14:53:56.584 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-09-12T14:53:56.584 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-09-12T14:53:56.584 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-09-12T14:53:56.585 INFO:tasks.ceph:Setting up mon.a... 2023-09-12T14:53:56.585 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-09-12T14:53:56.619 INFO:teuthology.orchestra.run.smithi131.stdout:creating /etc/ceph/ceph.keyring 2023-09-12T14:53:56.643 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-09-12T14:53:56.721 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-12T14:53:56.783 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.131'), ('mon.b', '172.21.15.187'), ('mon.c', '[v2:172.21.15.187:3301,v1:172.21.15.187:6790]')] 2023-09-12T14:53:56.783 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.131,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2023-09-12T14:53:56.784 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:53:56.785 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-09-12T14:53:56.830 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.131 --add b 172.21.15.187 --addv c '[v2:172.21.15.187:3301,v1:172.21.15.187:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-09-12T14:53:56.916 INFO:teuthology.orchestra.run.smithi131.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-09-12T14:53:56.917 INFO:teuthology.orchestra.run.smithi131.stdout:monmaptool: generated fsid 6efb91e0-47b4-40bb-ae12-7c7ea0e893df 2023-09-12T14:53:56.917 INFO:teuthology.orchestra.run.smithi131.stdout:setting min_mon_release = pacific 2023-09-12T14:53:56.917 INFO:teuthology.orchestra.run.smithi131.stdout:epoch 0 2023-09-12T14:53:56.918 INFO:teuthology.orchestra.run.smithi131.stdout:fsid 6efb91e0-47b4-40bb-ae12-7c7ea0e893df 2023-09-12T14:53:56.918 INFO:teuthology.orchestra.run.smithi131.stdout:last_changed 2023-09-12T14:53:56.917513+0000 2023-09-12T14:53:56.918 INFO:teuthology.orchestra.run.smithi131.stdout:created 2023-09-12T14:53:56.917513+0000 2023-09-12T14:53:56.918 INFO:teuthology.orchestra.run.smithi131.stdout:min_mon_release 16 (pacific) 2023-09-12T14:53:56.919 INFO:teuthology.orchestra.run.smithi131.stdout:election_strategy: 1 2023-09-12T14:53:56.919 INFO:teuthology.orchestra.run.smithi131.stdout:0: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.a 2023-09-12T14:53:56.919 INFO:teuthology.orchestra.run.smithi131.stdout:1: [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon.b 2023-09-12T14:53:56.919 INFO:teuthology.orchestra.run.smithi131.stdout:2: [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] mon.c 2023-09-12T14:53:56.920 INFO:teuthology.orchestra.run.smithi131.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-09-12T14:53:56.921 DEBUG:teuthology.orchestra.run.smithi131:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-09-12T14:53:56.966 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 6efb91e0-47b4-40bb-ae12-7c7ea0e893df... 2023-09-12T14:53:56.967 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-12T14:53:57.011 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-12T14:53:57.040 INFO:teuthology.orchestra.run.smithi131.stdout:[global] 2023-09-12T14:53:57.040 INFO:teuthology.orchestra.run.smithi131.stdout: chdir = "" 2023-09-12T14:53:57.040 INFO:teuthology.orchestra.run.smithi131.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-12T14:53:57.041 INFO:teuthology.orchestra.run.smithi131.stdout: auth supported = cephx 2023-09-12T14:53:57.041 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.041 INFO:teuthology.orchestra.run.smithi131.stdout: filestore xattr use omap = true 2023-09-12T14:53:57.041 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.041 INFO:teuthology.orchestra.run.smithi131.stdout: mon clock drift allowed = 1.000 2023-09-12T14:53:57.042 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.042 INFO:teuthology.orchestra.run.smithi131.stdout: osd crush chooseleaf type = 0 2023-09-12T14:53:57.042 INFO:teuthology.orchestra.run.smithi131.stdout: auth debug = true 2023-09-12T14:53:57.042 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.042 INFO:teuthology.orchestra.run.smithi131.stdout: ms die on old message = true 2023-09-12T14:53:57.043 INFO:teuthology.orchestra.run.smithi131.stdout: ms die on bug = true 2023-09-12T14:53:57.043 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.043 INFO:teuthology.orchestra.run.smithi131.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-12T14:53:57.043 INFO:teuthology.orchestra.run.smithi131.stdout: mon pg warn max object skew = 0 2023-09-12T14:53:57.043 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.043 INFO:teuthology.orchestra.run.smithi131.stdout: # disable pg_autoscaler by default for new pools 2023-09-12T14:53:57.044 INFO:teuthology.orchestra.run.smithi131.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-12T14:53:57.044 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.044 INFO:teuthology.orchestra.run.smithi131.stdout: osd pool default size = 2 2023-09-12T14:53:57.044 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.044 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd allow primary affinity = true 2023-09-12T14:53:57.044 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd allow pg remap = true 2023-09-12T14:53:57.045 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on legacy crush tunables = false 2023-09-12T14:53:57.045 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on crush straw calc version zero = false 2023-09-12T14:53:57.045 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on no sortbitwise = false 2023-09-12T14:53:57.045 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on osd down out interval zero = false 2023-09-12T14:53:57.045 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on too few osds = false 2023-09-12T14:53:57.046 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-12T14:53:57.046 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-12T14:53:57.046 INFO:teuthology.orchestra.run.smithi131.stdout: mon_allow_pool_size_one = true 2023-09-12T14:53:57.046 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.046 INFO:teuthology.orchestra.run.smithi131.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-12T14:53:57.047 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.047 INFO:teuthology.orchestra.run.smithi131.stdout: osd default data pool replay window = 5 2023-09-12T14:53:57.047 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.047 INFO:teuthology.orchestra.run.smithi131.stdout: mon allow pool delete = true 2023-09-12T14:53:57.047 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.047 INFO:teuthology.orchestra.run.smithi131.stdout: mon cluster log file level = debug 2023-09-12T14:53:57.048 INFO:teuthology.orchestra.run.smithi131.stdout: debug asserts on shutdown = true 2023-09-12T14:53:57.048 INFO:teuthology.orchestra.run.smithi131.stdout: mon health detail to clog = false 2023-09-12T14:53:57.048 INFO:teuthology.orchestra.run.smithi131.stdout: mon host = "172.21.15.131,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]" 2023-09-12T14:53:57.048 INFO:teuthology.orchestra.run.smithi131.stdout: fsid = 6efb91e0-47b4-40bb-ae12-7c7ea0e893df 2023-09-12T14:53:57.048 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.048 INFO:teuthology.orchestra.run.smithi131.stdout:[osd] 2023-09-12T14:53:57.049 INFO:teuthology.orchestra.run.smithi131.stdout: osd journal size = 100 2023-09-12T14:53:57.049 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.049 INFO:teuthology.orchestra.run.smithi131.stdout: osd scrub load threshold = 5.0 2023-09-12T14:53:57.049 INFO:teuthology.orchestra.run.smithi131.stdout: osd scrub max interval = 600 2023-09-12T14:53:57.049 INFO:teuthology.orchestra.run.smithi131.stdout: osd mclock profile = high_recovery_ops 2023-09-12T14:53:57.050 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.050 INFO:teuthology.orchestra.run.smithi131.stdout: osd recover clone overlap = true 2023-09-12T14:53:57.050 INFO:teuthology.orchestra.run.smithi131.stdout: osd recovery max chunk = 1048576 2023-09-12T14:53:57.050 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.050 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug shutdown = true 2023-09-12T14:53:57.051 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug op order = true 2023-09-12T14:53:57.051 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug verify stray on activate = true 2023-09-12T14:53:57.051 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.051 INFO:teuthology.orchestra.run.smithi131.stdout: osd open classes on start = true 2023-09-12T14:53:57.051 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug pg log writeout = true 2023-09-12T14:53:57.051 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.052 INFO:teuthology.orchestra.run.smithi131.stdout: osd deep scrub update digest min age = 30 2023-09-12T14:53:57.052 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.052 INFO:teuthology.orchestra.run.smithi131.stdout: osd map max advance = 10 2023-09-12T14:53:57.052 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.052 INFO:teuthology.orchestra.run.smithi131.stdout: journal zero on create = true 2023-09-12T14:53:57.052 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.053 INFO:teuthology.orchestra.run.smithi131.stdout: filestore ondisk finisher threads = 3 2023-09-12T14:53:57.053 INFO:teuthology.orchestra.run.smithi131.stdout: filestore apply finisher threads = 3 2023-09-12T14:53:57.053 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.053 INFO:teuthology.orchestra.run.smithi131.stdout: bdev debug aio = true 2023-09-12T14:53:57.053 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug misdirected ops = true 2023-09-12T14:53:57.053 INFO:teuthology.orchestra.run.smithi131.stdout: bdev async discard = True 2023-09-12T14:53:57.054 INFO:teuthology.orchestra.run.smithi131.stdout: bdev enable discard = True 2023-09-12T14:53:57.054 INFO:teuthology.orchestra.run.smithi131.stdout: bluestore allocator = bitmap 2023-09-12T14:53:57.054 INFO:teuthology.orchestra.run.smithi131.stdout: bluestore block size = 96636764160 2023-09-12T14:53:57.054 INFO:teuthology.orchestra.run.smithi131.stdout: bluestore fsck on mount = True 2023-09-12T14:53:57.054 INFO:teuthology.orchestra.run.smithi131.stdout: debug bluefs = 1/20 2023-09-12T14:53:57.054 INFO:teuthology.orchestra.run.smithi131.stdout: debug bluestore = 1/20 2023-09-12T14:53:57.055 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2023-09-12T14:53:57.055 INFO:teuthology.orchestra.run.smithi131.stdout: debug osd = 20 2023-09-12T14:53:57.055 INFO:teuthology.orchestra.run.smithi131.stdout: debug rocksdb = 4/10 2023-09-12T14:53:57.055 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd backfillfull_ratio = 0.85 2023-09-12T14:53:57.055 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd full ratio = 0.9 2023-09-12T14:53:57.055 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd nearfull ratio = 0.8 2023-09-12T14:53:57.056 INFO:teuthology.orchestra.run.smithi131.stdout: osd failsafe full ratio = 0.95 2023-09-12T14:53:57.056 INFO:teuthology.orchestra.run.smithi131.stdout: osd objectstore = bluestore 2023-09-12T14:53:57.056 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.056 INFO:teuthology.orchestra.run.smithi131.stdout:[mgr] 2023-09-12T14:53:57.056 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2023-09-12T14:53:57.057 INFO:teuthology.orchestra.run.smithi131.stdout: debug mgr = 20 2023-09-12T14:53:57.057 INFO:teuthology.orchestra.run.smithi131.stdout: debug mon = 20 2023-09-12T14:53:57.057 INFO:teuthology.orchestra.run.smithi131.stdout: debug auth = 20 2023-09-12T14:53:57.057 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min pgs per osd = 4 2023-09-12T14:53:57.057 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min bytes per osd = 10 2023-09-12T14:53:57.057 INFO:teuthology.orchestra.run.smithi131.stdout: mgr/telemetry/nag = false 2023-09-12T14:53:57.058 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.058 INFO:teuthology.orchestra.run.smithi131.stdout:[mon] 2023-09-12T14:53:57.058 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2023-09-12T14:53:57.058 INFO:teuthology.orchestra.run.smithi131.stdout: debug mon = 20 2023-09-12T14:53:57.058 INFO:teuthology.orchestra.run.smithi131.stdout: debug paxos = 20 2023-09-12T14:53:57.059 INFO:teuthology.orchestra.run.smithi131.stdout: debug auth = 20 2023-09-12T14:53:57.059 INFO:teuthology.orchestra.run.smithi131.stdout: mon data avail warn = 5 2023-09-12T14:53:57.059 INFO:teuthology.orchestra.run.smithi131.stdout: mon mgr mkfs grace = 240 2023-09-12T14:53:57.059 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min pgs per osd = 4 2023-09-12T14:53:57.059 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd reporter subtree level = osd 2023-09-12T14:53:57.059 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd prime pg temp = true 2023-09-12T14:53:57.060 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min bytes per osd = 10 2023-09-12T14:53:57.060 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.060 INFO:teuthology.orchestra.run.smithi131.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-12T14:53:57.060 INFO:teuthology.orchestra.run.smithi131.stdout: auth mon ticket ttl = 660 # 11m 2023-09-12T14:53:57.060 INFO:teuthology.orchestra.run.smithi131.stdout: auth service ticket ttl = 240 # 4m 2023-09-12T14:53:57.060 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.061 INFO:teuthology.orchestra.run.smithi131.stdout: # don't complain about insecure global_id in the test suite 2023-09-12T14:53:57.061 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-12T14:53:57.061 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-12T14:53:57.061 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.061 INFO:teuthology.orchestra.run.smithi131.stdout: # 1m isn't quite enough 2023-09-12T14:53:57.062 INFO:teuthology.orchestra.run.smithi131.stdout: mon_down_mkfs_grace = 2m 2023-09-12T14:53:57.062 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.062 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_filestore_osds = false 2023-09-12T14:53:57.062 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:53:57.062 INFO:teuthology.orchestra.run.smithi131.stdout:[client] 2023-09-12T14:53:57.063 INFO:teuthology.orchestra.run.smithi131.stdout: rgw cache enabled = true 2023-09-12T14:53:57.063 INFO:teuthology.orchestra.run.smithi131.stdout: rgw enable ops log = true 2023-09-12T14:53:57.063 INFO:teuthology.orchestra.run.smithi131.stdout: rgw enable usage log = true 2023-09-12T14:53:57.063 INFO:teuthology.orchestra.run.smithi131.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-12T14:53:57.063 INFO:teuthology.orchestra.run.smithi131.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-12T14:53:57.063 INFO:teuthology.orchestra.run.smithi131.stdout: pid file = "" 2023-09-12T14:53:57.064 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.a] 2023-09-12T14:53:57.064 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.b] 2023-09-12T14:53:57.064 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.c] 2023-09-12T14:53:57.064 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.a] 2023-09-12T14:53:57.064 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.c] 2023-09-12T14:53:57.064 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.b] 2023-09-12T14:53:57.065 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.d] 2023-09-12T14:53:57.066 INFO:teuthology.orchestra.run.smithi187.stdout:[global] 2023-09-12T14:53:57.066 INFO:teuthology.orchestra.run.smithi187.stdout: chdir = "" 2023-09-12T14:53:57.067 INFO:teuthology.orchestra.run.smithi187.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-12T14:53:57.067 INFO:teuthology.orchestra.run.smithi187.stdout: auth supported = cephx 2023-09-12T14:53:57.067 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.067 INFO:teuthology.orchestra.run.smithi187.stdout: filestore xattr use omap = true 2023-09-12T14:53:57.067 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.068 INFO:teuthology.orchestra.run.smithi187.stdout: mon clock drift allowed = 1.000 2023-09-12T14:53:57.068 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.068 INFO:teuthology.orchestra.run.smithi187.stdout: osd crush chooseleaf type = 0 2023-09-12T14:53:57.068 INFO:teuthology.orchestra.run.smithi187.stdout: auth debug = true 2023-09-12T14:53:57.068 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.068 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on old message = true 2023-09-12T14:53:57.069 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on bug = true 2023-09-12T14:53:57.069 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.069 INFO:teuthology.orchestra.run.smithi187.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-12T14:53:57.069 INFO:teuthology.orchestra.run.smithi187.stdout: mon pg warn max object skew = 0 2023-09-12T14:53:57.069 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.069 INFO:teuthology.orchestra.run.smithi187.stdout: # disable pg_autoscaler by default for new pools 2023-09-12T14:53:57.070 INFO:teuthology.orchestra.run.smithi187.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-12T14:53:57.070 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.070 INFO:teuthology.orchestra.run.smithi187.stdout: osd pool default size = 2 2023-09-12T14:53:57.070 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.070 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd allow primary affinity = true 2023-09-12T14:53:57.071 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd allow pg remap = true 2023-09-12T14:53:57.071 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on legacy crush tunables = false 2023-09-12T14:53:57.071 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on crush straw calc version zero = false 2023-09-12T14:53:57.071 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on no sortbitwise = false 2023-09-12T14:53:57.072 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on osd down out interval zero = false 2023-09-12T14:53:57.072 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on too few osds = false 2023-09-12T14:53:57.072 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-12T14:53:57.072 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-12T14:53:57.073 INFO:teuthology.orchestra.run.smithi187.stdout: mon_allow_pool_size_one = true 2023-09-12T14:53:57.073 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.073 INFO:teuthology.orchestra.run.smithi187.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-12T14:53:57.073 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.073 INFO:teuthology.orchestra.run.smithi187.stdout: osd default data pool replay window = 5 2023-09-12T14:53:57.074 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.074 INFO:teuthology.orchestra.run.smithi187.stdout: mon allow pool delete = true 2023-09-12T14:53:57.074 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.074 INFO:teuthology.orchestra.run.smithi187.stdout: mon cluster log file level = debug 2023-09-12T14:53:57.074 INFO:teuthology.orchestra.run.smithi187.stdout: debug asserts on shutdown = true 2023-09-12T14:53:57.074 INFO:teuthology.orchestra.run.smithi187.stdout: mon health detail to clog = false 2023-09-12T14:53:57.075 INFO:teuthology.orchestra.run.smithi187.stdout: mon host = "172.21.15.131,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]" 2023-09-12T14:53:57.075 INFO:teuthology.orchestra.run.smithi187.stdout: fsid = 6efb91e0-47b4-40bb-ae12-7c7ea0e893df 2023-09-12T14:53:57.075 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.075 INFO:teuthology.orchestra.run.smithi187.stdout:[osd] 2023-09-12T14:53:57.075 INFO:teuthology.orchestra.run.smithi187.stdout: osd journal size = 100 2023-09-12T14:53:57.076 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.076 INFO:teuthology.orchestra.run.smithi187.stdout: osd scrub load threshold = 5.0 2023-09-12T14:53:57.076 INFO:teuthology.orchestra.run.smithi187.stdout: osd scrub max interval = 600 2023-09-12T14:53:57.076 INFO:teuthology.orchestra.run.smithi187.stdout: osd mclock profile = high_recovery_ops 2023-09-12T14:53:57.076 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.076 INFO:teuthology.orchestra.run.smithi187.stdout: osd recover clone overlap = true 2023-09-12T14:53:57.077 INFO:teuthology.orchestra.run.smithi187.stdout: osd recovery max chunk = 1048576 2023-09-12T14:53:57.077 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.077 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug shutdown = true 2023-09-12T14:53:57.077 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug op order = true 2023-09-12T14:53:57.077 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug verify stray on activate = true 2023-09-12T14:53:57.077 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.078 INFO:teuthology.orchestra.run.smithi187.stdout: osd open classes on start = true 2023-09-12T14:53:57.078 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug pg log writeout = true 2023-09-12T14:53:57.078 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.078 INFO:teuthology.orchestra.run.smithi187.stdout: osd deep scrub update digest min age = 30 2023-09-12T14:53:57.079 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.079 INFO:teuthology.orchestra.run.smithi187.stdout: osd map max advance = 10 2023-09-12T14:53:57.079 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.079 INFO:teuthology.orchestra.run.smithi187.stdout: journal zero on create = true 2023-09-12T14:53:57.079 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.079 INFO:teuthology.orchestra.run.smithi187.stdout: filestore ondisk finisher threads = 3 2023-09-12T14:53:57.080 INFO:teuthology.orchestra.run.smithi187.stdout: filestore apply finisher threads = 3 2023-09-12T14:53:57.080 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.080 INFO:teuthology.orchestra.run.smithi187.stdout: bdev debug aio = true 2023-09-12T14:53:57.080 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug misdirected ops = true 2023-09-12T14:53:57.080 INFO:teuthology.orchestra.run.smithi187.stdout: bdev async discard = True 2023-09-12T14:53:57.081 INFO:teuthology.orchestra.run.smithi187.stdout: bdev enable discard = True 2023-09-12T14:53:57.081 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore allocator = bitmap 2023-09-12T14:53:57.081 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore block size = 96636764160 2023-09-12T14:53:57.081 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore fsck on mount = True 2023-09-12T14:53:57.081 INFO:teuthology.orchestra.run.smithi187.stdout: debug bluefs = 1/20 2023-09-12T14:53:57.081 INFO:teuthology.orchestra.run.smithi187.stdout: debug bluestore = 1/20 2023-09-12T14:53:57.082 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2023-09-12T14:53:57.082 INFO:teuthology.orchestra.run.smithi187.stdout: debug osd = 20 2023-09-12T14:53:57.082 INFO:teuthology.orchestra.run.smithi187.stdout: debug rocksdb = 4/10 2023-09-12T14:53:57.082 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd backfillfull_ratio = 0.85 2023-09-12T14:53:57.082 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd full ratio = 0.9 2023-09-12T14:53:57.083 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd nearfull ratio = 0.8 2023-09-12T14:53:57.083 INFO:teuthology.orchestra.run.smithi187.stdout: osd failsafe full ratio = 0.95 2023-09-12T14:53:57.083 INFO:teuthology.orchestra.run.smithi187.stdout: osd objectstore = bluestore 2023-09-12T14:53:57.083 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.083 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr] 2023-09-12T14:53:57.084 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2023-09-12T14:53:57.084 INFO:teuthology.orchestra.run.smithi187.stdout: debug mgr = 20 2023-09-12T14:53:57.084 INFO:teuthology.orchestra.run.smithi187.stdout: debug mon = 20 2023-09-12T14:53:57.084 INFO:teuthology.orchestra.run.smithi187.stdout: debug auth = 20 2023-09-12T14:53:57.084 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min pgs per osd = 4 2023-09-12T14:53:57.085 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min bytes per osd = 10 2023-09-12T14:53:57.085 INFO:teuthology.orchestra.run.smithi187.stdout: mgr/telemetry/nag = false 2023-09-12T14:53:57.085 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.085 INFO:teuthology.orchestra.run.smithi187.stdout:[mon] 2023-09-12T14:53:57.085 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2023-09-12T14:53:57.085 INFO:teuthology.orchestra.run.smithi187.stdout: debug mon = 20 2023-09-12T14:53:57.086 INFO:teuthology.orchestra.run.smithi187.stdout: debug paxos = 20 2023-09-12T14:53:57.086 INFO:teuthology.orchestra.run.smithi187.stdout: debug auth = 20 2023-09-12T14:53:57.086 INFO:teuthology.orchestra.run.smithi187.stdout: mon data avail warn = 5 2023-09-12T14:53:57.086 INFO:teuthology.orchestra.run.smithi187.stdout: mon mgr mkfs grace = 240 2023-09-12T14:53:57.086 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min pgs per osd = 4 2023-09-12T14:53:57.087 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd reporter subtree level = osd 2023-09-12T14:53:57.087 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd prime pg temp = true 2023-09-12T14:53:57.087 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min bytes per osd = 10 2023-09-12T14:53:57.087 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.087 INFO:teuthology.orchestra.run.smithi187.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-12T14:53:57.087 INFO:teuthology.orchestra.run.smithi187.stdout: auth mon ticket ttl = 660 # 11m 2023-09-12T14:53:57.088 INFO:teuthology.orchestra.run.smithi187.stdout: auth service ticket ttl = 240 # 4m 2023-09-12T14:53:57.088 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.088 INFO:teuthology.orchestra.run.smithi187.stdout: # don't complain about insecure global_id in the test suite 2023-09-12T14:53:57.088 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-12T14:53:57.088 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-12T14:53:57.088 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.089 INFO:teuthology.orchestra.run.smithi187.stdout: # 1m isn't quite enough 2023-09-12T14:53:57.089 INFO:teuthology.orchestra.run.smithi187.stdout: mon_down_mkfs_grace = 2m 2023-09-12T14:53:57.089 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.089 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_filestore_osds = false 2023-09-12T14:53:57.089 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-09-12T14:53:57.090 INFO:teuthology.orchestra.run.smithi187.stdout:[client] 2023-09-12T14:53:57.090 INFO:teuthology.orchestra.run.smithi187.stdout: rgw cache enabled = true 2023-09-12T14:53:57.090 INFO:teuthology.orchestra.run.smithi187.stdout: rgw enable ops log = true 2023-09-12T14:53:57.090 INFO:teuthology.orchestra.run.smithi187.stdout: rgw enable usage log = true 2023-09-12T14:53:57.090 INFO:teuthology.orchestra.run.smithi187.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-12T14:53:57.091 INFO:teuthology.orchestra.run.smithi187.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-12T14:53:57.091 INFO:teuthology.orchestra.run.smithi187.stdout: pid file = "" 2023-09-12T14:53:57.091 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.a] 2023-09-12T14:53:57.091 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.b] 2023-09-12T14:53:57.091 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.c] 2023-09-12T14:53:57.091 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.a] 2023-09-12T14:53:57.092 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.c] 2023-09-12T14:53:57.092 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.b] 2023-09-12T14:53:57.092 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.d] 2023-09-12T14:53:57.092 INFO:tasks.ceph:Creating admin key on mon.a... 2023-09-12T14:53:57.093 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-09-12T14:53:57.157 INFO:tasks.ceph:Copying monmap to all nodes... 2023-09-12T14:53:57.158 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:53:57.158 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-09-12T14:53:57.163 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:53:57.163 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-09-12T14:53:57.209 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi131.front.sepia.ceph.com 2023-09-12T14:53:57.210 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:53:57.210 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.keyring 2023-09-12T14:53:57.211 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-12T14:53:57.274 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:53:57.274 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-09-12T14:53:57.322 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi187.front.sepia.ceph.com 2023-09-12T14:53:57.323 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:53:57.323 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/ceph/ceph.keyring 2023-09-12T14:53:57.323 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-12T14:53:57.346 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:53:57.347 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-09-12T14:53:57.396 INFO:tasks.ceph:Setting up mon nodes... 2023-09-12T14:53:57.397 INFO:tasks.ceph:Setting up mgr nodes... 2023-09-12T14:53:57.397 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-09-12T14:53:57.439 INFO:teuthology.orchestra.run.smithi131.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-09-12T14:53:57.442 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-09-12T14:53:57.486 INFO:teuthology.orchestra.run.smithi187.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-09-12T14:53:57.506 INFO:tasks.ceph:Setting up mds nodes... 2023-09-12T14:53:57.506 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-09-12T14:53:57.547 INFO:teuthology.orchestra.run.smithi131.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-09-12T14:53:57.550 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-09-12T14:53:57.605 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-09-12T14:53:57.693 INFO:teuthology.orchestra.run.smithi131.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-09-12T14:53:57.695 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-09-12T14:53:57.749 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-09-12T14:53:57.794 INFO:teuthology.orchestra.run.smithi187.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-09-12T14:53:57.796 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-09-12T14:53:57.852 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-09-12T14:53:57.934 INFO:teuthology.orchestra.run.smithi187.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-09-12T14:53:57.937 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-09-12T14:53:57.991 INFO:tasks.ceph_client:Setting up client nodes... 2023-09-12T14:53:57.992 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-09-12T14:53:58.027 INFO:teuthology.orchestra.run.smithi131.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-09-12T14:53:58.039 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-09-12T14:53:58.112 INFO:teuthology.orchestra.run.smithi131.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-09-12T14:53:58.121 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-09-12T14:53:58.158 INFO:teuthology.orchestra.run.smithi187.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-09-12T14:53:58.168 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-09-12T14:53:58.245 INFO:teuthology.orchestra.run.smithi187.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-09-12T14:53:58.256 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-09-12T14:53:58.256 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi131.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi187.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-09-12T14:53:58.257 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-09-12T14:53:58.269 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-12T14:53:58.269 INFO:tasks.ceph:role: osd.0 2023-09-12T14:53:58.270 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi131.front.sepia.ceph.com 2023-09-12T14:53:58.270 DEBUG:teuthology.orchestra.run.smithi131:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-09-12T14:53:58.334 INFO:teuthology.orchestra.run.smithi131.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-09-12T14:53:58.334 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-12T14:53:58.335 INFO:teuthology.orchestra.run.smithi131.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-12T14:53:58.335 INFO:teuthology.orchestra.run.smithi131.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-12T14:53:58.335 INFO:teuthology.orchestra.run.smithi131.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-12T14:53:58.335 INFO:teuthology.orchestra.run.smithi131.stdout: = sunit=0 swidth=0 blks 2023-09-12T14:53:58.335 INFO:teuthology.orchestra.run.smithi131.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-12T14:53:58.335 INFO:teuthology.orchestra.run.smithi131.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-12T14:53:58.336 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-12T14:53:58.336 INFO:teuthology.orchestra.run.smithi131.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-12T14:53:58.786 INFO:teuthology.orchestra.run.smithi131.stdout:Discarding blocks...Done. 2023-09-12T14:53:58.788 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi131.front.sepia.ceph.com -o noatime 2023-09-12T14:53:58.788 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-09-12T14:53:58.881 DEBUG:teuthology.orchestra.run.smithi131:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-09-12T14:53:58.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T14:53:58.891 INFO:teuthology.orchestra.run.smithi131.stderr:sudo: /sbin/restorecon: command not found 2023-09-12T14:53:58.892 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-09-12T14:53:58.943 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-12T14:53:58.944 INFO:tasks.ceph:role: osd.1 2023-09-12T14:53:58.944 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi131.front.sepia.ceph.com 2023-09-12T14:53:58.944 DEBUG:teuthology.orchestra.run.smithi131:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-09-12T14:53:59.006 INFO:teuthology.orchestra.run.smithi131.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-09-12T14:53:59.006 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-12T14:53:59.006 INFO:teuthology.orchestra.run.smithi131.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-12T14:53:59.006 INFO:teuthology.orchestra.run.smithi131.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-12T14:53:59.007 INFO:teuthology.orchestra.run.smithi131.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-12T14:53:59.007 INFO:teuthology.orchestra.run.smithi131.stdout: = sunit=0 swidth=0 blks 2023-09-12T14:53:59.007 INFO:teuthology.orchestra.run.smithi131.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-12T14:53:59.007 INFO:teuthology.orchestra.run.smithi131.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-12T14:53:59.007 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-12T14:53:59.008 INFO:teuthology.orchestra.run.smithi131.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-12T14:53:59.479 INFO:teuthology.orchestra.run.smithi131.stdout:Discarding blocks...Done. 2023-09-12T14:53:59.481 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi131.front.sepia.ceph.com -o noatime 2023-09-12T14:53:59.482 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-09-12T14:53:59.622 DEBUG:teuthology.orchestra.run.smithi131:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-09-12T14:53:59.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T14:53:59.632 INFO:teuthology.orchestra.run.smithi131.stderr:sudo: /sbin/restorecon: command not found 2023-09-12T14:53:59.632 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-09-12T14:53:59.685 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-12T14:53:59.686 INFO:tasks.ceph:role: osd.2 2023-09-12T14:53:59.686 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi131.front.sepia.ceph.com 2023-09-12T14:53:59.686 DEBUG:teuthology.orchestra.run.smithi131:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-09-12T14:53:59.752 INFO:teuthology.orchestra.run.smithi131.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-09-12T14:53:59.753 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-12T14:53:59.753 INFO:teuthology.orchestra.run.smithi131.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-12T14:53:59.753 INFO:teuthology.orchestra.run.smithi131.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-12T14:53:59.753 INFO:teuthology.orchestra.run.smithi131.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-12T14:53:59.753 INFO:teuthology.orchestra.run.smithi131.stdout: = sunit=0 swidth=0 blks 2023-09-12T14:53:59.754 INFO:teuthology.orchestra.run.smithi131.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-12T14:53:59.754 INFO:teuthology.orchestra.run.smithi131.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-12T14:53:59.754 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-12T14:53:59.754 INFO:teuthology.orchestra.run.smithi131.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-12T14:54:00.234 INFO:teuthology.orchestra.run.smithi131.stdout:Discarding blocks...Done. 2023-09-12T14:54:00.236 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi131.front.sepia.ceph.com -o noatime 2023-09-12T14:54:00.237 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-09-12T14:54:00.378 DEBUG:teuthology.orchestra.run.smithi131:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-09-12T14:54:00.389 INFO:teuthology.orchestra.run.smithi131.stderr:sudo: /sbin/restorecon: command not found 2023-09-12T14:54:00.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T14:54:00.391 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-09-12T14:54:00.446 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-12T14:54:00.446 INFO:tasks.ceph:role: osd.3 2023-09-12T14:54:00.446 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi131.front.sepia.ceph.com 2023-09-12T14:54:00.447 DEBUG:teuthology.orchestra.run.smithi131:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-09-12T14:54:00.509 INFO:teuthology.orchestra.run.smithi131.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-09-12T14:54:00.510 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-12T14:54:00.510 INFO:teuthology.orchestra.run.smithi131.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-12T14:54:00.510 INFO:teuthology.orchestra.run.smithi131.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-12T14:54:00.510 INFO:teuthology.orchestra.run.smithi131.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-12T14:54:00.511 INFO:teuthology.orchestra.run.smithi131.stdout: = sunit=0 swidth=0 blks 2023-09-12T14:54:00.511 INFO:teuthology.orchestra.run.smithi131.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-12T14:54:00.511 INFO:teuthology.orchestra.run.smithi131.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-12T14:54:00.511 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-12T14:54:00.511 INFO:teuthology.orchestra.run.smithi131.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-12T14:54:00.938 INFO:teuthology.orchestra.run.smithi131.stdout:Discarding blocks...Done. 2023-09-12T14:54:00.940 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi131.front.sepia.ceph.com -o noatime 2023-09-12T14:54:00.941 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-09-12T14:54:01.031 DEBUG:teuthology.orchestra.run.smithi131:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-09-12T14:54:01.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T14:54:01.041 INFO:teuthology.orchestra.run.smithi131.stderr:sudo: /sbin/restorecon: command not found 2023-09-12T14:54:01.042 DEBUG:teuthology.orchestra.run.smithi131:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-12T14:54:01.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:01.138+0000 7f99be2d1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-09-12T14:54:01.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:01.138+0000 7f99be2d1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-09-12T14:54:01.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:01.138+0000 7f99be2d1640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-12T14:54:01.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:01.138+0000 7f99be2d1640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-12T14:54:01.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:01.138+0000 7f99be2d1640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-12T14:54:01.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:01.146+0000 7f99be2d1640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-09-12T14:54:04.158 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-09-12T14:54:04.172 DEBUG:teuthology.orchestra.run.smithi131:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-12T14:54:04.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:04.246+0000 7f43c75cc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-09-12T14:54:04.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:04.246+0000 7f43c75cc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-09-12T14:54:04.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:04.246+0000 7f43c75cc640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-12T14:54:04.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:04.246+0000 7f43c75cc640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-12T14:54:04.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:04.246+0000 7f43c75cc640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-12T14:54:04.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:04.250+0000 7f43c75cc640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-09-12T14:54:07.255 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-09-12T14:54:07.310 DEBUG:teuthology.orchestra.run.smithi131:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-12T14:54:07.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:07.386+0000 7feaa6a07640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-09-12T14:54:07.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:07.390+0000 7feaa6a07640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-09-12T14:54:07.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:07.390+0000 7feaa6a07640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-12T14:54:07.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:07.390+0000 7feaa6a07640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-12T14:54:07.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:07.390+0000 7feaa6a07640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-12T14:54:07.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:07.390+0000 7feaa6a07640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-09-12T14:54:10.393 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-09-12T14:54:10.450 DEBUG:teuthology.orchestra.run.smithi131:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-12T14:54:10.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:10.526+0000 7fd39982b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-09-12T14:54:10.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:10.530+0000 7fd39982b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-09-12T14:54:10.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:10.530+0000 7fd39982b640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-12T14:54:10.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:10.530+0000 7fd39982b640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-12T14:54:10.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:10.530+0000 7fd39982b640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-12T14:54:10.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:10.530+0000 7fd39982b640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-09-12T14:54:13.541 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-09-12T14:54:13.598 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-09-12T14:54:13.612 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-12T14:54:13.613 INFO:tasks.ceph:role: osd.4 2023-09-12T14:54:13.613 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi187.front.sepia.ceph.com 2023-09-12T14:54:13.613 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-09-12T14:54:13.676 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-09-12T14:54:13.677 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-12T14:54:13.677 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-12T14:54:13.677 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-12T14:54:13.677 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-12T14:54:13.678 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2023-09-12T14:54:13.678 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-12T14:54:13.678 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-12T14:54:13.678 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-12T14:54:13.678 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-12T14:54:14.137 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2023-09-12T14:54:14.139 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2023-09-12T14:54:14.140 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-09-12T14:54:14.245 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-09-12T14:54:14.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T14:54:14.257 INFO:teuthology.orchestra.run.smithi187.stderr:sudo: /sbin/restorecon: command not found 2023-09-12T14:54:14.257 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-09-12T14:54:14.311 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-12T14:54:14.312 INFO:tasks.ceph:role: osd.5 2023-09-12T14:54:14.312 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi187.front.sepia.ceph.com 2023-09-12T14:54:14.312 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-09-12T14:54:14.375 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-09-12T14:54:14.375 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-12T14:54:14.375 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-12T14:54:14.376 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-12T14:54:14.376 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-12T14:54:14.376 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2023-09-12T14:54:14.376 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-12T14:54:14.376 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-12T14:54:14.377 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-12T14:54:14.377 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-12T14:54:14.839 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2023-09-12T14:54:14.841 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2023-09-12T14:54:14.842 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-09-12T14:54:14.976 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-09-12T14:54:14.988 INFO:teuthology.orchestra.run.smithi187.stderr:sudo: /sbin/restorecon: command not found 2023-09-12T14:54:14.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T14:54:14.989 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-09-12T14:54:15.044 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-12T14:54:15.044 INFO:tasks.ceph:role: osd.6 2023-09-12T14:54:15.045 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi187.front.sepia.ceph.com 2023-09-12T14:54:15.045 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-09-12T14:54:15.107 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-09-12T14:54:15.108 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-12T14:54:15.108 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-12T14:54:15.108 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-12T14:54:15.109 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-12T14:54:15.109 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2023-09-12T14:54:15.109 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-12T14:54:15.109 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-12T14:54:15.110 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-12T14:54:15.110 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-12T14:54:15.586 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2023-09-12T14:54:15.588 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2023-09-12T14:54:15.588 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-09-12T14:54:15.733 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-09-12T14:54:15.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T14:54:15.745 INFO:teuthology.orchestra.run.smithi187.stderr:sudo: /sbin/restorecon: command not found 2023-09-12T14:54:15.746 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-09-12T14:54:15.797 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-12T14:54:15.798 INFO:tasks.ceph:role: osd.7 2023-09-12T14:54:15.798 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi187.front.sepia.ceph.com 2023-09-12T14:54:15.798 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-09-12T14:54:15.860 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-09-12T14:54:15.860 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-12T14:54:15.860 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-12T14:54:15.860 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-12T14:54:15.861 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-12T14:54:15.861 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2023-09-12T14:54:15.861 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-12T14:54:15.861 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-12T14:54:15.861 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-12T14:54:15.862 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-12T14:54:16.284 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2023-09-12T14:54:16.286 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2023-09-12T14:54:16.286 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-09-12T14:54:16.381 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-09-12T14:54:16.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T14:54:16.391 INFO:teuthology.orchestra.run.smithi187.stderr:sudo: /sbin/restorecon: command not found 2023-09-12T14:54:16.391 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-12T14:54:16.510 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:16.506+0000 7f74fba3f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-09-12T14:54:16.513 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:16.510+0000 7f74fba3f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-09-12T14:54:16.513 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:16.510+0000 7f74fba3f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-09-12T14:54:16.514 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:16.510+0000 7f74fba3f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-09-12T14:54:16.514 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:16.510+0000 7f74fba3f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-09-12T14:54:16.514 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:16.510+0000 7f74fba3f640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-09-12T14:54:19.525 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-09-12T14:54:19.539 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-12T14:54:19.621 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:19.618+0000 7f59dff3d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-09-12T14:54:19.622 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:19.618+0000 7f59dff3d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-09-12T14:54:19.622 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:19.618+0000 7f59dff3d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-09-12T14:54:19.622 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:19.618+0000 7f59dff3d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-09-12T14:54:19.623 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:19.618+0000 7f59dff3d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-09-12T14:54:19.623 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:19.618+0000 7f59dff3d640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-09-12T14:54:22.645 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-09-12T14:54:22.704 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-12T14:54:22.791 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:22.786+0000 7f6e9a8a0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-09-12T14:54:22.792 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:22.790+0000 7f6e9a8a0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-09-12T14:54:22.792 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:22.790+0000 7f6e9a8a0640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-09-12T14:54:22.792 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:22.790+0000 7f6e9a8a0640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-09-12T14:54:22.792 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:22.790+0000 7f6e9a8a0640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-09-12T14:54:22.793 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:22.790+0000 7f6e9a8a0640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-09-12T14:54:25.232 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-12T14:54:25.235 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-12T14:54:25.795 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-09-12T14:54:25.808 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-12T14:54:25.890 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:25.886+0000 7f3eb3674640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-09-12T14:54:25.891 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:25.886+0000 7f3eb3674640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-09-12T14:54:25.891 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:25.886+0000 7f3eb3674640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-09-12T14:54:25.891 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:25.886+0000 7f3eb3674640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-09-12T14:54:25.891 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:25.886+0000 7f3eb3674640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-09-12T14:54:25.892 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:25.890+0000 7f3eb3674640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-09-12T14:54:28.899 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-09-12T14:54:28.953 INFO:tasks.ceph:Reading keys from all nodes... 2023-09-12T14:54:28.953 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:54:28.953 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-09-12T14:54:28.967 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:54:28.967 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-09-12T14:54:29.021 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:54:29.021 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-09-12T14:54:29.076 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:54:29.076 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-09-12T14:54:29.128 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:54:29.128 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-09-12T14:54:29.181 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:54:29.182 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-09-12T14:54:29.236 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:54:29.237 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-09-12T14:54:29.289 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:54:29.289 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-09-12T14:54:29.303 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:54:29.303 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-09-12T14:54:29.355 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:54:29.355 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-09-12T14:54:29.410 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:54:29.411 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-09-12T14:54:29.463 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:54:29.464 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-09-12T14:54:29.519 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:54:29.520 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-09-12T14:54:29.574 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:54:29.574 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-09-12T14:54:29.626 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:54:29.627 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-09-12T14:54:29.633 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:54:29.633 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-09-12T14:54:29.682 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:54:29.682 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-09-12T14:54:29.688 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:54:29.688 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-09-12T14:54:29.736 INFO:tasks.ceph:Adding keys to all mons... 2023-09-12T14:54:29.736 DEBUG:teuthology.orchestra.run.smithi131:> sudo tee -a /etc/ceph/ceph.keyring 2023-09-12T14:54:29.738 DEBUG:teuthology.orchestra.run.smithi187:> sudo tee -a /etc/ceph/ceph.keyring 2023-09-12T14:54:29.782 INFO:teuthology.orchestra.run.smithi131.stdout:[mgr.y] 2023-09-12T14:54:29.782 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQCFewBliZ8/GhAAAMLjw4GVY9sK0KNQe7nJ/Q== 2023-09-12T14:54:29.782 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.a] 2023-09-12T14:54:29.782 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQCFewBl66m1IBAAACThNe9wpTrHBN/ZY493xw== 2023-09-12T14:54:29.783 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.c] 2023-09-12T14:54:29.783 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQCFewBlQAVaKRAA+H71HHnLvjttM1qDGTfy+g== 2023-09-12T14:54:29.783 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.0] 2023-09-12T14:54:29.783 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQCJewBl6FWSCBAAh9FESSSN2PT1cHiFqDJC3Q== 2023-09-12T14:54:29.783 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.1] 2023-09-12T14:54:29.784 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQCMewBliigYDxAA7a6XlL1w97mUPNj7kuqADA== 2023-09-12T14:54:29.784 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.2] 2023-09-12T14:54:29.784 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQCPewBlAjd9FxAA3sCdZikpCaVw/JlslRb2VQ== 2023-09-12T14:54:29.784 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.3] 2023-09-12T14:54:29.784 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQCSewBlr8HWHxAAj7UuvuDHcavTL0kZ6KIvqw== 2023-09-12T14:54:29.785 INFO:teuthology.orchestra.run.smithi131.stdout:[mgr.x] 2023-09-12T14:54:29.785 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQCFewBlX17uHBAAVt2YwHYj+9r1X8RBvmLEDA== 2023-09-12T14:54:29.785 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.b] 2023-09-12T14:54:29.785 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQCFewBlOENULxAAlYD82hfE/VldCdcujkgyZg== 2023-09-12T14:54:29.785 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.d] 2023-09-12T14:54:29.786 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQCFewBlE6u5NxAAp04RwzbaDxHv+CD99Q6fyQ== 2023-09-12T14:54:29.786 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.4] 2023-09-12T14:54:29.786 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQCYewBlRZ5qHhAAjba2RMH0atRum97TUk8ZbQ== 2023-09-12T14:54:29.786 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.5] 2023-09-12T14:54:29.787 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQCbewBlQv0QJRAAg4wipYCjrBl3Lda0R2F3bQ== 2023-09-12T14:54:29.787 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.6] 2023-09-12T14:54:29.787 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQCeewBlTf8zLxAAHlx1RrJ724NUYMWb+C4Viw== 2023-09-12T14:54:29.787 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.7] 2023-09-12T14:54:29.787 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQChewBlEb4aNRAAB9VRHeTwK6TnSxggszvCUw== 2023-09-12T14:54:29.788 INFO:teuthology.orchestra.run.smithi131.stdout:[client.0] 2023-09-12T14:54:29.788 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQCGewBlR6eoARAACciqMiYRB92w6zQMKfJGiA== 2023-09-12T14:54:29.788 INFO:teuthology.orchestra.run.smithi131.stdout:[client.2] 2023-09-12T14:54:29.788 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQCGewBl+Oa3BhAAHM8f/b08kfEhPZhkkySIRg== 2023-09-12T14:54:29.788 INFO:teuthology.orchestra.run.smithi131.stdout:[client.1] 2023-09-12T14:54:29.789 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQCGewBl6JxxCRAAe0pokP2AETTh2dX4lkCBmw== 2023-09-12T14:54:29.789 INFO:teuthology.orchestra.run.smithi131.stdout:[client.3] 2023-09-12T14:54:29.789 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQCGewBl+CSgDhAA+uCVLnyp7u4XUjxC5XdkJw== 2023-09-12T14:54:29.790 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr.y] 2023-09-12T14:54:29.790 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCFewBliZ8/GhAAAMLjw4GVY9sK0KNQe7nJ/Q== 2023-09-12T14:54:29.790 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.a] 2023-09-12T14:54:29.791 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCFewBl66m1IBAAACThNe9wpTrHBN/ZY493xw== 2023-09-12T14:54:29.791 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.c] 2023-09-12T14:54:29.791 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCFewBlQAVaKRAA+H71HHnLvjttM1qDGTfy+g== 2023-09-12T14:54:29.791 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.0] 2023-09-12T14:54:29.792 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCJewBl6FWSCBAAh9FESSSN2PT1cHiFqDJC3Q== 2023-09-12T14:54:29.792 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.1] 2023-09-12T14:54:29.792 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCMewBliigYDxAA7a6XlL1w97mUPNj7kuqADA== 2023-09-12T14:54:29.792 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.2] 2023-09-12T14:54:29.792 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCPewBlAjd9FxAA3sCdZikpCaVw/JlslRb2VQ== 2023-09-12T14:54:29.793 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.3] 2023-09-12T14:54:29.793 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCSewBlr8HWHxAAj7UuvuDHcavTL0kZ6KIvqw== 2023-09-12T14:54:29.793 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr.x] 2023-09-12T14:54:29.793 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCFewBlX17uHBAAVt2YwHYj+9r1X8RBvmLEDA== 2023-09-12T14:54:29.793 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.b] 2023-09-12T14:54:29.794 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCFewBlOENULxAAlYD82hfE/VldCdcujkgyZg== 2023-09-12T14:54:29.794 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.d] 2023-09-12T14:54:29.794 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCFewBlE6u5NxAAp04RwzbaDxHv+CD99Q6fyQ== 2023-09-12T14:54:29.794 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.4] 2023-09-12T14:54:29.794 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCYewBlRZ5qHhAAjba2RMH0atRum97TUk8ZbQ== 2023-09-12T14:54:29.795 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.5] 2023-09-12T14:54:29.795 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCbewBlQv0QJRAAg4wipYCjrBl3Lda0R2F3bQ== 2023-09-12T14:54:29.795 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.6] 2023-09-12T14:54:29.795 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCeewBlTf8zLxAAHlx1RrJ724NUYMWb+C4Viw== 2023-09-12T14:54:29.795 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.7] 2023-09-12T14:54:29.796 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQChewBlEb4aNRAAB9VRHeTwK6TnSxggszvCUw== 2023-09-12T14:54:29.796 INFO:teuthology.orchestra.run.smithi187.stdout:[client.0] 2023-09-12T14:54:29.796 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCGewBlR6eoARAACciqMiYRB92w6zQMKfJGiA== 2023-09-12T14:54:29.796 INFO:teuthology.orchestra.run.smithi187.stdout:[client.2] 2023-09-12T14:54:29.796 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCGewBl+Oa3BhAAHM8f/b08kfEhPZhkkySIRg== 2023-09-12T14:54:29.797 INFO:teuthology.orchestra.run.smithi187.stdout:[client.1] 2023-09-12T14:54:29.797 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCGewBl6JxxCRAAe0pokP2AETTh2dX4lkCBmw== 2023-09-12T14:54:29.797 INFO:teuthology.orchestra.run.smithi187.stdout:[client.3] 2023-09-12T14:54:29.797 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCGewBl+CSgDhAA+uCVLnyp7u4XUjxC5XdkJw== 2023-09-12T14:54:29.799 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-12T14:54:29.843 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-12T14:54:29.889 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-12T14:54:29.891 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-12T14:54:29.935 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-12T14:54:29.979 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-12T14:54:30.021 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-12T14:54:30.066 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-12T14:54:30.109 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-12T14:54:30.155 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-12T14:54:30.200 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-12T14:54:30.242 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-12T14:54:30.282 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-12T14:54:30.327 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-12T14:54:30.369 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-12T14:54:30.406 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-12T14:54:30.448 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-12T14:54:30.490 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-12T14:54:30.534 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-12T14:54:30.574 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-12T14:54:30.621 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-12T14:54:30.624 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-12T14:54:30.662 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-12T14:54:30.707 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-12T14:54:30.749 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-12T14:54:30.795 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-12T14:54:30.838 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-12T14:54:30.879 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-12T14:54:30.921 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-12T14:54:30.963 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-12T14:54:31.007 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-12T14:54:31.051 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-12T14:54:31.093 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-12T14:54:31.135 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-12T14:54:31.182 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-12T14:54:31.218 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-12T14:54:31.261 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-09-12T14:54:31.262 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-09-12T14:54:31.312 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-12T14:54:31.406 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-09-12T14:54:31.417 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-09-12T14:54:31.431 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-12T14:54:31.531 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-09-12T14:54:31.543 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-09-12T14:54:31.597 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-12T14:54:31.691 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-09-12T14:54:31.702 DEBUG:teuthology.orchestra.run.smithi131:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-09-12T14:54:31.705 DEBUG:teuthology.orchestra.run.smithi187:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-09-12T14:54:31.747 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-09-12T14:54:31.747 INFO:tasks.ceph.mon.a:Restarting daemon 2023-09-12T14:54:31.748 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-09-12T14:54:31.754 INFO:tasks.ceph.mon.a:Started 2023-09-12T14:54:31.754 INFO:tasks.ceph.mon.b:Restarting daemon 2023-09-12T14:54:31.755 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-09-12T14:54:31.793 INFO:tasks.ceph.mon.b:Started 2023-09-12T14:54:31.793 INFO:tasks.ceph.mon.c:Restarting daemon 2023-09-12T14:54:31.794 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-09-12T14:54:31.796 INFO:tasks.ceph.mon.c:Started 2023-09-12T14:54:31.796 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2023-09-12T14:54:31.797 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2023-09-12T14:54:31.799 INFO:tasks.ceph:setting config [client] debug_client = 20 2023-09-12T14:54:31.800 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set client debug_client 20 2023-09-12T14:54:31.845 INFO:tasks.ceph:setting config [client] debug_ms = 1 2023-09-12T14:54:31.846 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set client debug_ms 1 2023-09-12T14:54:31.890 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2023-09-12T14:54:31.890 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2023-09-12T14:54:31.933 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2023-09-12T14:54:31.934 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2023-09-12T14:54:31.977 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2023-09-12T14:54:31.978 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mds debug_mds 20 2023-09-12T14:54:32.022 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2023-09-12T14:54:32.022 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2023-09-12T14:54:32.066 INFO:tasks.ceph:setting config [mds] debug_ms = 1 2023-09-12T14:54:32.066 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mds debug_ms 1 2023-09-12T14:54:32.113 INFO:tasks.ceph:setting config [mds] mds_bal_fragment_size_max = 10000 2023-09-12T14:54:32.114 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mds mds_bal_fragment_size_max 10000 2023-09-12T14:54:32.161 INFO:tasks.ceph:setting config [mds] mds_bal_merge_size = 5 2023-09-12T14:54:32.162 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mds mds_bal_merge_size 5 2023-09-12T14:54:32.209 INFO:tasks.ceph:setting config [mds] mds_bal_split_bits = 3 2023-09-12T14:54:32.210 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mds mds_bal_split_bits 3 2023-09-12T14:54:32.254 INFO:tasks.ceph:setting config [mds] mds_bal_split_size = 100 2023-09-12T14:54:32.254 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mds mds_bal_split_size 100 2023-09-12T14:54:32.298 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2023-09-12T14:54:32.299 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2023-09-12T14:54:32.342 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2023-09-12T14:54:32.342 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2023-09-12T14:54:32.387 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2023-09-12T14:54:32.387 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2023-09-12T14:54:32.422 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2023-09-12T14:54:32.422 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2023-09-12T14:54:32.451 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2023-09-12T14:54:32.451 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2023-09-12T14:54:32.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.466+0000 7f4a5d1d6640 1 -- 172.21.15.131:0/3793724885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a58139350 msgr2=0x7f4a581397d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.466+0000 7f4a5d1d6640 1 --2- 172.21.15.131:0/3793724885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a58139350 0x7f4a581397d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4a580ce100 tx=0x7f4a50006720 comp rx=0 tx=0).stop 2023-09-12T14:54:32.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.466+0000 7f4a5d1d6640 1 -- 172.21.15.131:0/3793724885 shutdown_connections 2023-09-12T14:54:32.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.466+0000 7f4a5d1d6640 1 --2- 172.21.15.131:0/3793724885 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a58137ac0 0x7f4a5815a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.466+0000 7f4a5d1d6640 1 --2- 172.21.15.131:0/3793724885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a58139350 0x7f4a581397d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.466+0000 7f4a5d1d6640 1 --2- 172.21.15.131:0/3793724885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a440039d0 0x7f4a580b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.466+0000 7f4a5d1d6640 1 -- 172.21.15.131:0/3793724885 >> 172.21.15.131:0/3793724885 conn(0x7f4a5806a520 msgr2=0x7f4a581413b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:32.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.466+0000 7f4a5d1d6640 1 -- 172.21.15.131:0/3793724885 shutdown_connections 2023-09-12T14:54:32.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.466+0000 7f4a5d1d6640 1 -- 172.21.15.131:0/3793724885 wait complete. 2023-09-12T14:54:32.474 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2023-09-12T14:54:32.474 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2023-09-12T14:54:32.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.486+0000 7f98c654d640 1 -- 172.21.15.131:0/54010504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98ac0047d0 msgr2=0x7f98ac004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.486+0000 7f98c654d640 1 --2- 172.21.15.131:0/54010504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98ac0047d0 0x7f98ac004c50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f98bc006ea0 tx=0x7f98bc01f5d0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.486+0000 7f98c654d640 1 -- 172.21.15.131:0/54010504 shutdown_connections 2023-09-12T14:54:32.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.486+0000 7f98c654d640 1 --2- 172.21.15.131:0/54010504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98ac0047d0 0x7f98ac004c50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.486+0000 7f98c654d640 1 -- 172.21.15.131:0/54010504 >> 172.21.15.131:0/54010504 conn(0x7f98c00ccb90 msgr2=0x7f98c00ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:32.493 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2023-09-12T14:54:32.494 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2023-09-12T14:54:32.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.486+0000 7f98c654d640 1 -- 172.21.15.131:0/54010504 shutdown_connections 2023-09-12T14:54:32.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.486+0000 7f98c654d640 1 -- 172.21.15.131:0/54010504 wait complete. 2023-09-12T14:54:32.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.490+0000 7f98c654d640 1 Processor -- start 2023-09-12T14:54:32.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.490+0000 7f98c654d640 1 -- start start 2023-09-12T14:54:32.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.490+0000 7f98c654d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98ac0047d0 0x7f98c006fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.490+0000 7f98c654d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98c00b0370 0x7f98c00b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.490+0000 7f98c654d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98c0155630 0x7f98c0155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.490+0000 7f98c654d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c01551a0 con 0x7f98ac0047d0 2023-09-12T14:54:32.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.490+0000 7f98c654d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c0154590 con 0x7f98c00b0370 2023-09-12T14:54:32.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.490+0000 7f98c654d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c0154870 con 0x7f98c0155630 2023-09-12T14:54:32.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.490+0000 7f98c554b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98ac0047d0 0x7f98c006fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.490+0000 7f98c554b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98ac0047d0 0x7f98c006fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56056/0 (socket says 172.21.15.131:56056) 2023-09-12T14:54:32.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.490+0000 7f98c554b640 1 -- 172.21.15.131:0/3170519598 learned_addr learned my addr 172.21.15.131:0/3170519598 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:32.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.490+0000 7f98c4d4a640 1 --2- 172.21.15.131:0/3170519598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98c00b0370 0x7f98c00b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.494+0000 7f98c554b640 1 -- 172.21.15.131:0/3170519598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98c0155630 msgr2=0x7f98c0155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.494+0000 7f98c554b640 1 --2- 172.21.15.131:0/3170519598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98c0155630 0x7f98c0155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.494+0000 7f98c554b640 1 -- 172.21.15.131:0/3170519598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98c00b0370 msgr2=0x7f98c00b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.494+0000 7f98c554b640 1 --2- 172.21.15.131:0/3170519598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98c00b0370 0x7f98c00b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.494+0000 7f98c554b640 1 -- 172.21.15.131:0/3170519598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c0154b50 con 0x7f98ac0047d0 2023-09-12T14:54:32.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.494+0000 7f98c554b640 1 --2- 172.21.15.131:0/3170519598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98ac0047d0 0x7f98c006fce0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f98b0001000 tx=0x7f98b0003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:32.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.494+0000 7f98b5ffb640 1 -- 172.21.15.131:0/3170519598 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b000f980 con 0x7f98ac0047d0 2023-09-12T14:54:32.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.494+0000 7f98c654d640 1 -- 172.21.15.131:0/3170519598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c0154e30 con 0x7f98ac0047d0 2023-09-12T14:54:32.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.494+0000 7f98c654d640 1 -- 172.21.15.131:0/3170519598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c00703f0 con 0x7f98ac0047d0 2023-09-12T14:54:32.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.494+0000 7f98b5ffb640 1 -- 172.21.15.131:0/3170519598 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f98b000fae0 con 0x7f98ac0047d0 2023-09-12T14:54:32.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.494+0000 7f98b5ffb640 1 -- 172.21.15.131:0/3170519598 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b0011430 con 0x7f98ac0047d0 2023-09-12T14:54:32.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.494+0000 7f98b5ffb640 1 -- 172.21.15.131:0/3170519598 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f98b00115d0 con 0x7f98ac0047d0 2023-09-12T14:54:32.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.494+0000 7f98b5ffb640 1 -- 172.21.15.131:0/3170519598 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f98b0010a00 con 0x7f98ac0047d0 2023-09-12T14:54:32.509 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2023-09-12T14:54:32.510 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2023-09-12T14:54:32.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.502+0000 7f98c654d640 1 -- 172.21.15.131:0/3170519598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a0005260 con 0x7f98ac0047d0 2023-09-12T14:54:32.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.506+0000 7f98b5ffb640 1 -- 172.21.15.131:0/3170519598 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f98b0020060 con 0x7f98ac0047d0 2023-09-12T14:54:32.513 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2023-09-12T14:54:32.513 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2023-09-12T14:54:32.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.526+0000 7fb2aac36640 1 -- 172.21.15.131:0/2004283198 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) v1 -- 0x7fb27c0052c0 con 0x7fb2a415c130 2023-09-12T14:54:32.531 DEBUG:tasks.ceph:set 21 configs 2023-09-12T14:54:32.531 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi131') 2023-09-12T14:54:32.532 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi131') 2023-09-12T14:54:32.532 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi131') 2023-09-12T14:54:32.532 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi131') 2023-09-12T14:54:32.533 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi131') 2023-09-12T14:54:32.533 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi131') 2023-09-12T14:54:32.533 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi131') 2023-09-12T14:54:32.533 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '1'], check_status=True, hostname='smithi131') 2023-09-12T14:54:32.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.530+0000 7fb2a17fa640 1 -- 172.21.15.131:0/2004283198 <== mon.0 v2:172.21.15.131:3300/0 12 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v9)=0 v9) v1 ==== 106+0+0 (secure 0 0 0) 0x7fb29c013c70 con 0x7fb2a415c130 2023-09-12T14:54:32.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.554+0000 7fb2aac36640 1 -- 172.21.15.131:0/2004283198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2a415c130 msgr2=0x7fb2a406c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.554+0000 7fb2aac36640 1 --2- 172.21.15.131:0/2004283198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2a415c130 0x7fb2a406c580 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb29c00b900 tx=0x7fb29c00bdc0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.554+0000 7fb2aac36640 1 -- 172.21.15.131:0/2004283198 shutdown_connections 2023-09-12T14:54:32.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.554+0000 7fb2aac36640 1 --2- 172.21.15.131:0/2004283198 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2a406fd60 0x7fb2a4137560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.554+0000 7fb2aac36640 1 --2- 172.21.15.131:0/2004283198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2a415c130 0x7fb2a406c580 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.554+0000 7fb2aac36640 1 --2- 172.21.15.131:0/2004283198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2a406f980 0x7fb2a413eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.554+0000 7fb2aac36640 1 -- 172.21.15.131:0/2004283198 >> 172.21.15.131:0/2004283198 conn(0x7fb2a40ae6b0 msgr2=0x7fb2a40ad9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:32.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.554+0000 7fb2aac36640 1 -- 172.21.15.131:0/2004283198 shutdown_connections 2023-09-12T14:54:32.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.554+0000 7fb2aac36640 1 -- 172.21.15.131:0/2004283198 wait complete. 2023-09-12T14:54:32.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.562+0000 7f009989d640 1 -- 172.21.15.131:0/791672094 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) v1 -- 0x7f0074005170 con 0x7f0094157c80 2023-09-12T14:54:32.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.578+0000 7f0083fff640 1 -- 172.21.15.131:0/791672094 <== mon.2 v2:172.21.15.187:3301/0 11 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v10)=0 v10) v1 ==== 127+0+0 (secure 0 0 0) 0x7f008802a030 con 0x7f0094157c80 2023-09-12T14:54:32.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.586+0000 7f9d46475640 1 -- 172.21.15.131:0/3310617572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d400ccde0 msgr2=0x7f9d400ac150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.586+0000 7f9d46475640 1 --2- 172.21.15.131:0/3310617572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d400ccde0 0x7f9d400ac150 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9d400ab500 tx=0x7f9d3801cc80 comp rx=0 tx=0).stop 2023-09-12T14:54:32.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.586+0000 7f9d46475640 1 -- 172.21.15.131:0/3310617572 shutdown_connections 2023-09-12T14:54:32.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.586+0000 7f9d46475640 1 --2- 172.21.15.131:0/3310617572 >> v2:172.21.15.131:3300/0 conn(0x7f9d400caab0 0x7f9d400cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.586+0000 7f9d46475640 1 --2- 172.21.15.131:0/3310617572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d400ccde0 0x7f9d400ac150 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.586+0000 7f9d46475640 1 -- 172.21.15.131:0/3310617572 >> 172.21.15.131:0/3310617572 conn(0x7f9d40069640 msgr2=0x7f9d40069a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:32.594 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi131') 2023-09-12T14:54:32.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.590+0000 7fb22e181640 1 -- 172.21.15.131:0/504243025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2280c8d00 msgr2=0x7fb2280cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.590+0000 7fb22e181640 1 --2- 172.21.15.131:0/504243025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2280c8d00 0x7fb2280cc430 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb22806e810 tx=0x7fb21801c860 comp rx=0 tx=0).stop 2023-09-12T14:54:32.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.590+0000 7f009989d640 1 -- 172.21.15.131:0/791672094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0094157c80 msgr2=0x7f0094154f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.590+0000 7f009989d640 1 --2- 172.21.15.131:0/791672094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0094157c80 0x7f0094154f20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f008800d7a0 tx=0x7f008800dc60 comp rx=0 tx=0).stop 2023-09-12T14:54:32.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.590+0000 7f9d46475640 1 -- 172.21.15.131:0/3310617572 shutdown_connections 2023-09-12T14:54:32.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.590+0000 7fb22e181640 1 -- 172.21.15.131:0/504243025 shutdown_connections 2023-09-12T14:54:32.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.590+0000 7fb22e181640 1 --2- 172.21.15.131:0/504243025 >> v2:172.21.15.187:3300/0 conn(0x7fb22815b9e0 0x7fb228158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.590+0000 7fb22e181640 1 --2- 172.21.15.131:0/504243025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2280c8d00 0x7fb2280cc430 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.590+0000 7fb22e181640 1 -- 172.21.15.131:0/504243025 >> 172.21.15.131:0/504243025 conn(0x7fb2280ccb90 msgr2=0x7fb2280ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:32.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.590+0000 7fb22e181640 1 -- 172.21.15.131:0/504243025 shutdown_connections 2023-09-12T14:54:32.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.590+0000 7f9d46475640 1 -- 172.21.15.131:0/3310617572 wait complete. 2023-09-12T14:54:32.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7f9d46475640 1 Processor -- start 2023-09-12T14:54:32.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.590+0000 7fb22e181640 1 -- 172.21.15.131:0/504243025 wait complete. 2023-09-12T14:54:32.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7fb22e181640 1 Processor -- start 2023-09-12T14:54:32.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7f9d46475640 1 -- start start 2023-09-12T14:54:32.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7f9d46475640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d4006ee20 0x7f9d40200bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7f9d46475640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d400caab0 0x7f9d4006b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7f9d46475640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d400ccde0 0x7f9d40153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7f9d46475640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4006c640 con 0x7f9d400ccde0 2023-09-12T14:54:32.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7f9d46475640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d400a8170 con 0x7f9d4006ee20 2023-09-12T14:54:32.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7f9d46475640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d40151ed0 con 0x7f9d400caab0 2023-09-12T14:54:32.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7fb22e181640 1 -- start start 2023-09-12T14:54:32.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7fb22e181640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2280c8d00 0x7fb22814d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7fb22e181640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb22815b590 0x7fb2281f4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7fb22e181640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb22815b9e0 0x7fb2281f4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7fb22e181640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2281503f0 con 0x7fb22815b9e0 2023-09-12T14:54:32.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7fb22e181640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2281f6960 con 0x7fb22815b590 2023-09-12T14:54:32.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7fb22e181640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2281f6c40 con 0x7fb2280c8d00 2023-09-12T14:54:32.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7f009989d640 1 -- 172.21.15.131:0/791672094 shutdown_connections 2023-09-12T14:54:32.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7f9d44a12640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d400ccde0 0x7f9d40153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7fb22d980640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb22815b9e0 0x7fb2281f4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7fb22d980640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb22815b9e0 0x7fb2281f4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56084/0 (socket says 172.21.15.131:56084) 2023-09-12T14:54:32.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7fb22d980640 1 -- 172.21.15.131:0/3898334944 learned_addr learned my addr 172.21.15.131:0/3898334944 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:32.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7f009989d640 1 --2- 172.21.15.131:0/791672094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0094157c80 0x7f0094154f20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7f9d44a12640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d400ccde0 0x7f9d40153230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56080/0 (socket says 172.21.15.131:56080) 2023-09-12T14:54:32.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7f9d44a12640 1 -- 172.21.15.131:0/3434997586 learned_addr learned my addr 172.21.15.131:0/3434997586 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:32.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7f009989d640 1 --2- 172.21.15.131:0/791672094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f009415c0a0 0x7f00940c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7f009989d640 1 --2- 172.21.15.131:0/791672094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00941578a0 0x7f009413c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7f009989d640 1 -- 172.21.15.131:0/791672094 >> 172.21.15.131:0/791672094 conn(0x7f0094065680 msgr2=0x7f0094150950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:32.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7f9d3f7fe640 1 --2- 172.21.15.131:0/3434997586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d400caab0 0x7f9d4006b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7f009989d640 1 -- 172.21.15.131:0/791672094 shutdown_connections 2023-09-12T14:54:32.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7f9d3ffff640 1 --2- 172.21.15.131:0/3434997586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d4006ee20 0x7f9d40200bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7fb22d17f640 1 --2- 172.21.15.131:0/3898334944 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2280c8d00 0x7fb22814d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7fb22c97e640 1 --2- 172.21.15.131:0/3898334944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb22815b590 0x7fb2281f4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7f009989d640 1 -- 172.21.15.131:0/791672094 wait complete. 2023-09-12T14:54:32.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7f9d44a12640 1 -- 172.21.15.131:0/3434997586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d400caab0 msgr2=0x7f9d4006b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7f9d44a12640 1 --2- 172.21.15.131:0/3434997586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d400caab0 0x7f9d4006b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7f9d44a12640 1 -- 172.21.15.131:0/3434997586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d4006ee20 msgr2=0x7f9d40200bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7f9d44a12640 1 --2- 172.21.15.131:0/3434997586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d4006ee20 0x7f9d40200bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7f9d44a12640 1 -- 172.21.15.131:0/3434997586 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d380098e0 con 0x7f9d400ccde0 2023-09-12T14:54:32.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.594+0000 7f9d44a12640 1 --2- 172.21.15.131:0/3434997586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d400ccde0 0x7f9d40153230 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9d3400c410 tx=0x7f9d3400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:32.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.598+0000 7fb22d980640 1 -- 172.21.15.131:0/3898334944 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2280c8d00 msgr2=0x7fb22814d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.598+0000 7fb22d980640 1 --2- 172.21.15.131:0/3898334944 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2280c8d00 0x7fb22814d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.598+0000 7fb22d980640 1 -- 172.21.15.131:0/3898334944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb22815b590 msgr2=0x7fb2281f4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.598+0000 7fb22d980640 1 --2- 172.21.15.131:0/3898334944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb22815b590 0x7fb2281f4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.598+0000 7fb22d980640 1 -- 172.21.15.131:0/3898334944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb218009560 con 0x7fb22815b9e0 2023-09-12T14:54:32.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.598+0000 7fb22d980640 1 --2- 172.21.15.131:0/3898334944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb22815b9e0 0x7fb2281f4ea0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb22000e970 tx=0x7fb22000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:32.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.606+0000 7f9d3cff9640 1 -- 172.21.15.131:0/3434997586 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d34003600 con 0x7f9d400ccde0 2023-09-12T14:54:32.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.606+0000 7fb21dffb640 1 -- 172.21.15.131:0/3898334944 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb220004d30 con 0x7fb22815b9e0 2023-09-12T14:54:32.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.606+0000 7f9d46475640 1 -- 172.21.15.131:0/3434997586 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d4006ce10 con 0x7f9d400ccde0 2023-09-12T14:54:32.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.606+0000 7f9d46475640 1 -- 172.21.15.131:0/3434997586 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d4006cb30 con 0x7f9d400ccde0 2023-09-12T14:54:32.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.606+0000 7f9d3cff9640 1 -- 172.21.15.131:0/3434997586 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9d34003780 con 0x7f9d400ccde0 2023-09-12T14:54:32.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.606+0000 7fb21dffb640 1 -- 172.21.15.131:0/3898334944 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb22000cb20 con 0x7fb22815b9e0 2023-09-12T14:54:32.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.606+0000 7fb21dffb640 1 -- 172.21.15.131:0/3898334944 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb220005020 con 0x7fb22815b9e0 2023-09-12T14:54:32.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.606+0000 7f9d3cff9640 1 -- 172.21.15.131:0/3434997586 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d34003bd0 con 0x7f9d400ccde0 2023-09-12T14:54:32.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.606+0000 7f9d3cff9640 1 -- 172.21.15.131:0/3434997586 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9d3401c740 con 0x7f9d400ccde0 2023-09-12T14:54:32.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.606+0000 7f9d3cff9640 1 -- 172.21.15.131:0/3434997586 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9d340128d0 con 0x7f9d400ccde0 2023-09-12T14:54:32.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.606+0000 7f9d46475640 1 -- 172.21.15.131:0/3434997586 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d20005260 con 0x7f9d400ccde0 2023-09-12T14:54:32.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.606+0000 7fb22e181640 1 -- 172.21.15.131:0/3898334944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2281f6f20 con 0x7fb22815b9e0 2023-09-12T14:54:32.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.606+0000 7fb22e181640 1 -- 172.21.15.131:0/3898334944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2281f74e0 con 0x7fb22815b9e0 2023-09-12T14:54:32.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.610+0000 7f9d3cff9640 1 -- 172.21.15.131:0/3434997586 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f9d34017070 con 0x7f9d400ccde0 2023-09-12T14:54:32.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.610+0000 7fb21dffb640 1 -- 172.21.15.131:0/3898334944 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb220005480 con 0x7fb22815b9e0 2023-09-12T14:54:32.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.610+0000 7fb21dffb640 1 -- 172.21.15.131:0/3898334944 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb220011a00 con 0x7fb22815b9e0 2023-09-12T14:54:32.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.618+0000 7fb22e181640 1 -- 172.21.15.131:0/3898334944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2281505f0 con 0x7fb22815b9e0 2023-09-12T14:54:32.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.622+0000 7fb21dffb640 1 -- 172.21.15.131:0/3898334944 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fb2281505f0 con 0x7fb22815b9e0 2023-09-12T14:54:32.634 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi131') 2023-09-12T14:54:32.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.630+0000 7ff69bfff640 1 -- 172.21.15.131:0/813884405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff69c15eab0 msgr2=0x7ff69c0cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.630+0000 7ff69bfff640 1 --2- 172.21.15.131:0/813884405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff69c15eab0 0x7ff69c0cc200 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff690009900 tx=0x7ff69001c8e0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.642+0000 7ff69bfff640 1 -- 172.21.15.131:0/813884405 shutdown_connections 2023-09-12T14:54:32.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.642+0000 7ff69bfff640 1 --2- 172.21.15.131:0/813884405 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff69c0ab750 0x7ff69c0abbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.642+0000 7ff69bfff640 1 --2- 172.21.15.131:0/813884405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff69c15eab0 0x7ff69c0cc200 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.642+0000 7ff69bfff640 1 -- 172.21.15.131:0/813884405 >> 172.21.15.131:0/813884405 conn(0x7ff69c0ccb90 msgr2=0x7ff69c0ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:32.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.650+0000 7ff69bfff640 1 -- 172.21.15.131:0/813884405 shutdown_connections 2023-09-12T14:54:32.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.650+0000 7ff69bfff640 1 -- 172.21.15.131:0/813884405 wait complete. 2023-09-12T14:54:32.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.650+0000 7f98c654d640 1 -- 172.21.15.131:0/3170519598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) v1 -- 0x7f98a0005880 con 0x7f98ac0047d0 2023-09-12T14:54:32.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.650+0000 7ff69bfff640 1 Processor -- start 2023-09-12T14:54:32.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.650+0000 7ff69bfff640 1 -- start start 2023-09-12T14:54:32.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.650+0000 7ff69bfff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff69c0ab750 0x7ff69c14d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.650+0000 7ff69bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff69c0cc740 0x7ff69c14da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.650+0000 7ff69bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff69c15eab0 0x7ff69c15a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.650+0000 7ff69bfff640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c0705c0 con 0x7ff69c0cc740 2023-09-12T14:54:32.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.650+0000 7ff69bfff640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c14df40 con 0x7ff69c15eab0 2023-09-12T14:54:32.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.650+0000 7ff69bfff640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c06fa40 con 0x7ff69c0ab750 2023-09-12T14:54:32.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.654+0000 7ff69affd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff69c0ab750 0x7ff69c14d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.654+0000 7ff69affd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff69c0ab750 0x7ff69c14d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:33908/0 (socket says 172.21.15.131:33908) 2023-09-12T14:54:32.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.654+0000 7ff69affd640 1 -- 172.21.15.131:0/3250183637 learned_addr learned my addr 172.21.15.131:0/3250183637 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:32.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.654+0000 7ff69b7fe640 1 --2- 172.21.15.131:0/3250183637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff69c15eab0 0x7ff69c15a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.654+0000 7ff69affd640 1 -- 172.21.15.131:0/3250183637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff69c15eab0 msgr2=0x7ff69c15a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.654+0000 7ff69affd640 1 --2- 172.21.15.131:0/3250183637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff69c15eab0 0x7ff69c15a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.654+0000 7ff69affd640 1 -- 172.21.15.131:0/3250183637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff69c0cc740 msgr2=0x7ff69c14da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.654+0000 7ff69affd640 1 --2- 172.21.15.131:0/3250183637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff69c0cc740 0x7ff69c14da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.654+0000 7ff69affd640 1 -- 172.21.15.131:0/3250183637 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff690009560 con 0x7ff69c0ab750 2023-09-12T14:54:32.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.654+0000 7ff69b7fe640 1 --2- 172.21.15.131:0/3250183637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff69c15eab0 0x7ff69c15a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T14:54:32.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.654+0000 7ff69affd640 1 --2- 172.21.15.131:0/3250183637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff69c0ab750 0x7ff69c14d4c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff69001a510 tx=0x7ff6900a5780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:32.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.654+0000 7f98b5ffb640 1 -- 172.21.15.131:0/3170519598 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v11)=0 v11) v1 ==== 116+0+0 (secure 0 0 0) 0x7f98b0015070 con 0x7f98ac0047d0 2023-09-12T14:54:32.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.662+0000 7f98c654d640 1 -- 172.21.15.131:0/3170519598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98ac0047d0 msgr2=0x7f98c006fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.662+0000 7f98c654d640 1 --2- 172.21.15.131:0/3170519598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98ac0047d0 0x7f98c006fce0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f98b0001000 tx=0x7f98b0003a90 comp rx=0 tx=0).stop 2023-09-12T14:54:32.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.666+0000 7ff683fff640 1 -- 172.21.15.131:0/3250183637 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6900bb070 con 0x7ff69c0ab750 2023-09-12T14:54:32.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.666+0000 7ff69bfff640 1 -- 172.21.15.131:0/3250183637 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff69c06f760 con 0x7ff69c0ab750 2023-09-12T14:54:32.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.666+0000 7ff69bfff640 1 -- 172.21.15.131:0/3250183637 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff69c070a90 con 0x7ff69c0ab750 2023-09-12T14:54:32.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.666+0000 7ff683fff640 1 -- 172.21.15.131:0/3250183637 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff690004130 con 0x7ff69c0ab750 2023-09-12T14:54:32.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.666+0000 7ff683fff640 1 -- 172.21.15.131:0/3250183637 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690003600 con 0x7ff69c0ab750 2023-09-12T14:54:32.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.666+0000 7f98c654d640 1 -- 172.21.15.131:0/3170519598 shutdown_connections 2023-09-12T14:54:32.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.666+0000 7f98c654d640 1 --2- 172.21.15.131:0/3170519598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98c0155630 0x7f98c0155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.666+0000 7ff683fff640 1 -- 172.21.15.131:0/3250183637 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff6900b4040 con 0x7ff69c0ab750 2023-09-12T14:54:32.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.666+0000 7f98c654d640 1 --2- 172.21.15.131:0/3170519598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98c00b0370 0x7f98c00b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.666+0000 7f98c654d640 1 --2- 172.21.15.131:0/3170519598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98ac0047d0 0x7f98c006fce0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.666+0000 7f98c654d640 1 -- 172.21.15.131:0/3170519598 >> 172.21.15.131:0/3170519598 conn(0x7f98c00ccb90 msgr2=0x7f98c015b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:32.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.666+0000 7ff683fff640 1 -- 172.21.15.131:0/3250183637 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff6900ad8c0 con 0x7ff69c0ab750 2023-09-12T14:54:32.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.674+0000 7f98c654d640 1 -- 172.21.15.131:0/3170519598 shutdown_connections 2023-09-12T14:54:32.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.674+0000 7f98c654d640 1 -- 172.21.15.131:0/3170519598 wait complete. 2023-09-12T14:54:32.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.674+0000 7fcab6231640 1 -- 172.21.15.131:0/3715026778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcab00cd250 msgr2=0x7fcab006f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.674+0000 7fcab6231640 1 --2- 172.21.15.131:0/3715026778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcab00cd250 0x7fcab006f9a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009cb0 tx=0x7fcaa801cbd0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.674+0000 7fcab6231640 1 -- 172.21.15.131:0/3715026778 shutdown_connections 2023-09-12T14:54:32.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.674+0000 7fcab6231640 1 --2- 172.21.15.131:0/3715026778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcab00cd250 0x7fcab006f9a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.674+0000 7fcab6231640 1 -- 172.21.15.131:0/3715026778 >> 172.21.15.131:0/3715026778 conn(0x7fcab0069640 msgr2=0x7fcab0069a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:32.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.674+0000 7fcab6231640 1 -- 172.21.15.131:0/3715026778 shutdown_connections 2023-09-12T14:54:32.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.674+0000 7fcab6231640 1 -- 172.21.15.131:0/3715026778 wait complete. 2023-09-12T14:54:32.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.674+0000 7fcab6231640 1 Processor -- start 2023-09-12T14:54:32.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.674+0000 7fcab6231640 1 -- start start 2023-09-12T14:54:32.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.678+0000 7fcab6231640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcab00ccde0 0x7fcab0155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.678+0000 7fcab6231640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcab00cd250 0x7fcab014f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.678+0000 7fcab6231640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcab014eba0 0x7fcab014f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.678+0000 7fcab6231640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab014fec0 con 0x7fcab00cd250 2023-09-12T14:54:32.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.678+0000 7fcab6231640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab014fbe0 con 0x7fcab014eba0 2023-09-12T14:54:32.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.678+0000 7fcab6231640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab014c340 con 0x7fcab00ccde0 2023-09-12T14:54:32.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.678+0000 7fcaaeffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcab00cd250 0x7fcab014f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.678+0000 7fcaaffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcab014eba0 0x7fcab014f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.678+0000 7fcaaffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcab014eba0 0x7fcab014f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:50806/0 (socket says 172.21.15.131:50806) 2023-09-12T14:54:32.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.678+0000 7fcaaffff640 1 -- 172.21.15.131:0/2715769937 learned_addr learned my addr 172.21.15.131:0/2715769937 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:32.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.678+0000 7fcaaf7fe640 1 --2- 172.21.15.131:0/2715769937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcab00ccde0 0x7fcab0155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.678+0000 7fcaaffff640 1 -- 172.21.15.131:0/2715769937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcab00ccde0 msgr2=0x7fcab0155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.678+0000 7fcaaffff640 1 --2- 172.21.15.131:0/2715769937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcab00ccde0 0x7fcab0155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.678+0000 7fcaaffff640 1 -- 172.21.15.131:0/2715769937 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcab00cd250 msgr2=0x7fcab014f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.678+0000 7fcaaffff640 1 --2- 172.21.15.131:0/2715769937 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcab00cd250 0x7fcab014f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.678+0000 7fcaaffff640 1 -- 172.21.15.131:0/2715769937 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa80098e0 con 0x7fcab014eba0 2023-09-12T14:54:32.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.678+0000 7fcaaeffd640 1 --2- 172.21.15.131:0/2715769937 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcab00cd250 0x7fcab014f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T14:54:32.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.678+0000 7fcaaffff640 1 --2- 172.21.15.131:0/2715769937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcab014eba0 0x7fcab014f5d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80b0040 tx=0x7fcaa80045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:32.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.690+0000 7fca97fff640 1 -- 172.21.15.131:0/2715769937 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa80a5c80 con 0x7fcab014eba0 2023-09-12T14:54:32.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.690+0000 7fca97fff640 1 -- 172.21.15.131:0/2715769937 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcaa80ad6a0 con 0x7fcab014eba0 2023-09-12T14:54:32.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.690+0000 7fca97fff640 1 -- 172.21.15.131:0/2715769937 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa80adab0 con 0x7fcab014eba0 2023-09-12T14:54:32.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.690+0000 7ff69bfff640 1 -- 172.21.15.131:0/3250183637 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff67c005260 con 0x7ff69c0ab750 2023-09-12T14:54:32.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.690+0000 7fcab6231640 1 -- 172.21.15.131:0/2715769937 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab015c410 con 0x7fcab014eba0 2023-09-12T14:54:32.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.690+0000 7fcab6231640 1 -- 172.21.15.131:0/2715769937 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab014c900 con 0x7fcab014eba0 2023-09-12T14:54:32.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.690+0000 7fca97fff640 1 -- 172.21.15.131:0/2715769937 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcaa80b4040 con 0x7fcab014eba0 2023-09-12T14:54:32.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.690+0000 7fca97fff640 1 -- 172.21.15.131:0/2715769937 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcaa80b6c30 con 0x7fcab014eba0 2023-09-12T14:54:32.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.694+0000 7ff683fff640 1 -- 172.21.15.131:0/3250183637 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7ff69c0c8740 con 0x7ff69c0ab750 2023-09-12T14:54:32.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.702+0000 7fcab6231640 1 -- 172.21.15.131:0/2715769937 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab014c180 con 0x7fcab014eba0 2023-09-12T14:54:32.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.706+0000 7fca97fff640 1 -- 172.21.15.131:0/2715769937 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fcaa80b2070 con 0x7fcab014eba0 2023-09-12T14:54:32.718 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi131') 2023-09-12T14:54:32.750 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.746+0000 7fa940228640 1 -- 172.21.15.131:0/1488814288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa93806ee20 msgr2=0x7fa938154e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.750 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.746+0000 7fa940228640 1 --2- 172.21.15.131:0/1488814288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa93806ee20 0x7fa938154e50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa9380ab500 tx=0x7fa93001cb60 comp rx=0 tx=0).stop 2023-09-12T14:54:32.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.754+0000 7fa940228640 1 -- 172.21.15.131:0/1488814288 shutdown_connections 2023-09-12T14:54:32.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.754+0000 7fa940228640 1 --2- 172.21.15.131:0/1488814288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa93806ee20 0x7fa938154e50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.754+0000 7fa940228640 1 --2- 172.21.15.131:0/1488814288 >> v2:172.21.15.187:3300/0 conn(0x7fa9380ccde0 0x7fa9380ac150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.754+0000 7fa940228640 1 -- 172.21.15.131:0/1488814288 >> 172.21.15.131:0/1488814288 conn(0x7fa938069640 msgr2=0x7fa938069a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:32.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.754+0000 7fa940228640 1 -- 172.21.15.131:0/1488814288 shutdown_connections 2023-09-12T14:54:32.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.754+0000 7fa940228640 1 -- 172.21.15.131:0/1488814288 wait complete. 2023-09-12T14:54:32.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.754+0000 7fa940228640 1 Processor -- start 2023-09-12T14:54:32.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.754+0000 7fa940228640 1 -- start start 2023-09-12T14:54:32.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.754+0000 7fa940228640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa93806ee20 0x7fa938144d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.754+0000 7fa940228640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9380caab0 0x7fa938150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.754+0000 7fa940228640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9380ccde0 0x7fa9380caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.754+0000 7fa940228640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9381f5760 con 0x7fa9380caab0 2023-09-12T14:54:32.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.754+0000 7fa940228640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9381f5a40 con 0x7fa93806ee20 2023-09-12T14:54:32.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.754+0000 7fa940228640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9381f5d20 con 0x7fa9380ccde0 2023-09-12T14:54:32.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.754+0000 7fa93d7c3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9380caab0 0x7fa938150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.754+0000 7fa93d7c3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9380caab0 0x7fa938150010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56130/0 (socket says 172.21.15.131:56130) 2023-09-12T14:54:32.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.754+0000 7fa93d7c3640 1 -- 172.21.15.131:0/2309740765 learned_addr learned my addr 172.21.15.131:0/2309740765 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:32.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.758+0000 7fa93e7c5640 1 --2- 172.21.15.131:0/2309740765 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9380ccde0 0x7fa9380caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.758+0000 7fa93dfc4640 1 --2- 172.21.15.131:0/2309740765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa93806ee20 0x7fa938144d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.758+0000 7fa93d7c3640 1 -- 172.21.15.131:0/2309740765 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9380ccde0 msgr2=0x7fa9380caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.758+0000 7fa93d7c3640 1 --2- 172.21.15.131:0/2309740765 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9380ccde0 0x7fa9380caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.758+0000 7fa93d7c3640 1 -- 172.21.15.131:0/2309740765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa93806ee20 msgr2=0x7fa938144d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.758+0000 7fa93d7c3640 1 --2- 172.21.15.131:0/2309740765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa93806ee20 0x7fa938144d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.758+0000 7fa93d7c3640 1 -- 172.21.15.131:0/2309740765 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9300098e0 con 0x7fa9380caab0 2023-09-12T14:54:32.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.758+0000 7fa93d7c3640 1 --2- 172.21.15.131:0/2309740765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9380caab0 0x7fa938150010 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa934009ec0 tx=0x7fa93400c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:32.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.758+0000 7fa92e7fc640 1 -- 172.21.15.131:0/2309740765 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93400ce10 con 0x7fa9380caab0 2023-09-12T14:54:32.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.758+0000 7fa940228640 1 -- 172.21.15.131:0/2309740765 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9381f6000 con 0x7fa9380caab0 2023-09-12T14:54:32.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.758+0000 7fa940228640 1 -- 172.21.15.131:0/2309740765 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9381452b0 con 0x7fa9380caab0 2023-09-12T14:54:32.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.758+0000 7f5b9edd5640 1 -- 172.21.15.131:0/3303434666 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b980ccde0 msgr2=0x7f5b980ac150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.758+0000 7f5b9edd5640 1 --2- 172.21.15.131:0/3303434666 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b980ccde0 0x7f5b980ac150 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5b88009870 tx=0x7f5b8801c230 comp rx=0 tx=0).stop 2023-09-12T14:54:32.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.758+0000 7f5b9edd5640 1 -- 172.21.15.131:0/3303434666 shutdown_connections 2023-09-12T14:54:32.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.758+0000 7f5b9edd5640 1 --2- 172.21.15.131:0/3303434666 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b980ccde0 0x7f5b980ac150 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.758+0000 7f5b9edd5640 1 -- 172.21.15.131:0/3303434666 >> 172.21.15.131:0/3303434666 conn(0x7f5b98069640 msgr2=0x7f5b98069a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:32.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.758+0000 7f5b9edd5640 1 -- 172.21.15.131:0/3303434666 shutdown_connections 2023-09-12T14:54:32.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.758+0000 7f5b9edd5640 1 -- 172.21.15.131:0/3303434666 wait complete. 2023-09-12T14:54:32.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.758+0000 7fa940228640 1 -- 172.21.15.131:0/2309740765 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa93804e110 con 0x7fa9380caab0 2023-09-12T14:54:32.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.762+0000 7f5b9edd5640 1 Processor -- start 2023-09-12T14:54:32.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.762+0000 7f9d46475640 1 -- 172.21.15.131:0/3434997586 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) v1 -- 0x7f9d20005880 con 0x7f9d400ccde0 2023-09-12T14:54:32.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.762+0000 7f5b9edd5640 1 -- start start 2023-09-12T14:54:32.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.762+0000 7f5b9edd5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b9806ee20 0x7f5b9813f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.762+0000 7f5b9edd5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b980caab0 0x7f5b980b0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.762+0000 7f5b9edd5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b980ccde0 0x7f5b980adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.762+0000 7f5b9edd5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b980b0fe0 con 0x7f5b980caab0 2023-09-12T14:54:32.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.762+0000 7f5b9edd5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b980b12c0 con 0x7f5b980ccde0 2023-09-12T14:54:32.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.762+0000 7f5b9edd5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b980b15a0 con 0x7f5b9806ee20 2023-09-12T14:54:32.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.762+0000 7fa92e7fc640 1 -- 172.21.15.131:0/2309740765 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa934004110 con 0x7fa9380caab0 2023-09-12T14:54:32.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.762+0000 7fa92e7fc640 1 -- 172.21.15.131:0/2309740765 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa934003600 con 0x7fa9380caab0 2023-09-12T14:54:32.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.762+0000 7fa92e7fc640 1 -- 172.21.15.131:0/2309740765 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa934003b20 con 0x7fa9380caab0 2023-09-12T14:54:32.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.762+0000 7f5b97fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b980caab0 0x7f5b980b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.762+0000 7f5b97fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b980caab0 0x7f5b980b0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56146/0 (socket says 172.21.15.131:56146) 2023-09-12T14:54:32.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.762+0000 7f5b97fff640 1 -- 172.21.15.131:0/3906470640 learned_addr learned my addr 172.21.15.131:0/3906470640 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:32.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.762+0000 7fa92e7fc640 1 -- 172.21.15.131:0/2309740765 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa93401d410 con 0x7fa9380caab0 2023-09-12T14:54:32.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.762+0000 7f5b9cb71640 1 --2- 172.21.15.131:0/3906470640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b9806ee20 0x7f5b9813f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.762+0000 7f5b9d372640 1 --2- 172.21.15.131:0/3906470640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b980ccde0 0x7f5b980adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.776 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.762+0000 7f5b97fff640 1 -- 172.21.15.131:0/3906470640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b9806ee20 msgr2=0x7f5b9813f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.776 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.762+0000 7f5b97fff640 1 --2- 172.21.15.131:0/3906470640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b9806ee20 0x7f5b9813f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.776 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.762+0000 7f5b97fff640 1 -- 172.21.15.131:0/3906470640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b980ccde0 msgr2=0x7f5b980adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.776 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.762+0000 7f5b97fff640 1 --2- 172.21.15.131:0/3906470640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b980ccde0 0x7f5b980adb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.762+0000 7f5b97fff640 1 -- 172.21.15.131:0/3906470640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b880094a0 con 0x7f5b980caab0 2023-09-12T14:54:32.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.762+0000 7f5b97fff640 1 --2- 172.21.15.131:0/3906470640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b980caab0 0x7f5b980b0570 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00ee40 tx=0x7f5b8c00c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:32.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.762+0000 7f5b957fa640 1 -- 172.21.15.131:0/3906470640 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c014620 con 0x7f5b980caab0 2023-09-12T14:54:32.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.762+0000 7fa92e7fc640 1 -- 172.21.15.131:0/2309740765 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fa934018030 con 0x7fa9380caab0 2023-09-12T14:54:32.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.766+0000 7f5b957fa640 1 -- 172.21.15.131:0/3906470640 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5b8c014870 con 0x7f5b980caab0 2023-09-12T14:54:32.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.766+0000 7f5b957fa640 1 -- 172.21.15.131:0/3906470640 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c014d00 con 0x7f5b980caab0 2023-09-12T14:54:32.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.766+0000 7f5b9edd5640 1 -- 172.21.15.131:0/3906470640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b980b1880 con 0x7f5b980caab0 2023-09-12T14:54:32.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.766+0000 7f5b9edd5640 1 -- 172.21.15.131:0/3906470640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b98154cd0 con 0x7f5b980caab0 2023-09-12T14:54:32.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.766+0000 7f5b957fa640 1 -- 172.21.15.131:0/3906470640 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5b8c01d700 con 0x7f5b980caab0 2023-09-12T14:54:32.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.766+0000 7f5b957fa640 1 -- 172.21.15.131:0/3906470640 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5b8c01c6a0 con 0x7f5b980caab0 2023-09-12T14:54:32.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.766+0000 7f5b9edd5640 1 -- 172.21.15.131:0/3906470640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b9804e110 con 0x7f5b980caab0 2023-09-12T14:54:32.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.770+0000 7f9d3cff9640 1 -- 172.21.15.131:0/3434997586 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v12)=0 v12) v1 ==== 116+0+0 (secure 0 0 0) 0x7f9d34003d30 con 0x7f9d400ccde0 2023-09-12T14:54:32.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.770+0000 7f9d46475640 1 -- 172.21.15.131:0/3434997586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d400ccde0 msgr2=0x7f9d40153230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.770+0000 7f9d46475640 1 --2- 172.21.15.131:0/3434997586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d400ccde0 0x7f9d40153230 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9d3400c410 tx=0x7f9d3400c8d0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.770+0000 7f9d46475640 1 -- 172.21.15.131:0/3434997586 shutdown_connections 2023-09-12T14:54:32.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.770+0000 7f9d46475640 1 --2- 172.21.15.131:0/3434997586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d400caab0 0x7f9d4006b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.770+0000 7f9d46475640 1 --2- 172.21.15.131:0/3434997586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d400ccde0 0x7f9d40153230 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.770+0000 7f9d46475640 1 --2- 172.21.15.131:0/3434997586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d4006ee20 0x7f9d40200bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.770+0000 7f9d46475640 1 -- 172.21.15.131:0/3434997586 >> 172.21.15.131:0/3434997586 conn(0x7f9d40069640 msgr2=0x7f9d40169560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:32.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.774+0000 7f9d46475640 1 -- 172.21.15.131:0/3434997586 shutdown_connections 2023-09-12T14:54:32.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.774+0000 7f9d46475640 1 -- 172.21.15.131:0/3434997586 wait complete. 2023-09-12T14:54:32.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f19ebd11640 1 -- 172.21.15.131:0/576863950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19e40af8b0 msgr2=0x7f19e40cbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f19ebd11640 1 --2- 172.21.15.131:0/576863950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19e40af8b0 0x7f19e40cbd60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009bc0 tx=0x7f19dc01cb00 comp rx=0 tx=0).stop 2023-09-12T14:54:32.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f19ebd11640 1 -- 172.21.15.131:0/576863950 shutdown_connections 2023-09-12T14:54:32.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f19ebd11640 1 --2- 172.21.15.131:0/576863950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19e40af8b0 0x7f19e40cbd60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f19ebd11640 1 --2- 172.21.15.131:0/576863950 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19e415e140 0x7f19e40cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f19ebd11640 1 -- 172.21.15.131:0/576863950 >> 172.21.15.131:0/576863950 conn(0x7f19e40cd4d0 msgr2=0x7f19e40cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:32.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f19ebd11640 1 -- 172.21.15.131:0/576863950 shutdown_connections 2023-09-12T14:54:32.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f19ebd11640 1 -- 172.21.15.131:0/576863950 wait complete. 2023-09-12T14:54:32.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f19ebd11640 1 Processor -- start 2023-09-12T14:54:32.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f4f21785640 1 -- 172.21.15.131:0/1932243014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f1c0c2960 msgr2=0x7f4f1c0c2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f4f21785640 1 --2- 172.21.15.131:0/1932243014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f1c0c2960 0x7f4f1c0c2d70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0c95c0 tx=0x7f4f1401cce0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f4f21785640 1 -- 172.21.15.131:0/1932243014 shutdown_connections 2023-09-12T14:54:32.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f4f21785640 1 --2- 172.21.15.131:0/1932243014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f1c0c2960 0x7f4f1c0c2d70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f4f21785640 1 --2- 172.21.15.131:0/1932243014 >> v2:172.21.15.131:3300/0 conn(0x7f4f1c156fb0 0x7f4f1c06fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f4f21785640 1 -- 172.21.15.131:0/1932243014 >> 172.21.15.131:0/1932243014 conn(0x7f4f1c069240 msgr2=0x7f4f1c069650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:32.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f4f21785640 1 -- 172.21.15.131:0/1932243014 shutdown_connections 2023-09-12T14:54:32.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f4f21785640 1 -- 172.21.15.131:0/1932243014 wait complete. 2023-09-12T14:54:32.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f4f21785640 1 Processor -- start 2023-09-12T14:54:32.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f4f21785640 1 -- start start 2023-09-12T14:54:32.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f4f21785640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f1c0708b0 0x7f4f1c147650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f4f21785640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f1c0c2960 0x7f4f1c164f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f4f21785640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f1c156fb0 0x7f4f1c161130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f4f21785640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c1600d0 con 0x7f4f1c0708b0 2023-09-12T14:54:32.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f4f21785640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c160370 con 0x7f4f1c156fb0 2023-09-12T14:54:32.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f4f21785640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c160650 con 0x7f4f1c0c2960 2023-09-12T14:54:32.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f4f1bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f1c0708b0 0x7f4f1c147650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f4f1bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f1c0708b0 0x7f4f1c147650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56196/0 (socket says 172.21.15.131:56196) 2023-09-12T14:54:32.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f4f1bfff640 1 -- 172.21.15.131:0/3476647534 learned_addr learned my addr 172.21.15.131:0/3476647534 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:32.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f3d96c41640 1 -- 172.21.15.131:0/4190700659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d90158c70 msgr2=0x7f3d900afe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f3d96c41640 1 --2- 172.21.15.131:0/4190700659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d90158c70 0x7f3d900afe10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3d900cd9f0 tx=0x7f3d8401e540 comp rx=0 tx=0).stop 2023-09-12T14:54:32.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f3d96c41640 1 -- 172.21.15.131:0/4190700659 shutdown_connections 2023-09-12T14:54:32.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f3d96c41640 1 --2- 172.21.15.131:0/4190700659 >> v2:172.21.15.131:3300/0 conn(0x7f3d900b0350 0x7f3d900ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f3d96c41640 1 --2- 172.21.15.131:0/4190700659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d90158c70 0x7f3d900afe10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f3d96c41640 1 --2- 172.21.15.131:0/4190700659 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d9015bc30 0x7f3d9015c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f3d96c41640 1 -- 172.21.15.131:0/4190700659 >> 172.21.15.131:0/4190700659 conn(0x7f3d90069640 msgr2=0x7f3d90069a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:32.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f3d96c41640 1 -- 172.21.15.131:0/4190700659 shutdown_connections 2023-09-12T14:54:32.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f3d96c41640 1 -- 172.21.15.131:0/4190700659 wait complete. 2023-09-12T14:54:32.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f3d96c41640 1 Processor -- start 2023-09-12T14:54:32.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f3d96c41640 1 -- start start 2023-09-12T14:54:32.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f3d96c41640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d900b0350 0x7f3d901f7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f3d96c41640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d90158c70 0x7f3d902017f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f3d96c41640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d9015bc30 0x7f3d901515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f3d949dd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d900b0350 0x7f3d901f7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f3d949dd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d900b0350 0x7f3d901f7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:33990/0 (socket says 172.21.15.131:33990) 2023-09-12T14:54:32.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f3d8ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d90158c70 0x7f3d902017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f3d8ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d90158c70 0x7f3d902017f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:50910/0 (socket says 172.21.15.131:50910) 2023-09-12T14:54:32.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f3d949dd640 1 -- 172.21.15.131:0/2909785642 learned_addr learned my addr 172.21.15.131:0/2909785642 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:32.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f3d96c41640 1 -- 172.21.15.131:0/2909785642 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d90143000 con 0x7f3d9015bc30 2023-09-12T14:54:32.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f3d96c41640 1 -- 172.21.15.131:0/2909785642 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d90201d30 con 0x7f3d90158c70 2023-09-12T14:54:32.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f3d96c41640 1 -- 172.21.15.131:0/2909785642 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d90151b20 con 0x7f3d900b0350 2023-09-12T14:54:32.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f3d951de640 1 --2- 172.21.15.131:0/2909785642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d9015bc30 0x7f3d901515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f3d949dd640 1 -- 172.21.15.131:0/2909785642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d90158c70 msgr2=0x7f3d902017f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f3d949dd640 1 --2- 172.21.15.131:0/2909785642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d90158c70 0x7f3d902017f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f3d949dd640 1 -- 172.21.15.131:0/2909785642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d9015bc30 msgr2=0x7f3d901515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f3d949dd640 1 --2- 172.21.15.131:0/2909785642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d9015bc30 0x7f3d901515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f3d949dd640 1 -- 172.21.15.131:0/2909785642 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8400b6b0 con 0x7f3d900b0350 2023-09-12T14:54:32.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f3d949dd640 1 --2- 172.21.15.131:0/2909785642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d900b0350 0x7f3d901f7980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c00e7a0 tx=0x7f3d7c00ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:32.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f3d8d7fa640 1 -- 172.21.15.131:0/2909785642 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c00ccc0 con 0x7f3d900b0350 2023-09-12T14:54:32.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.770+0000 7f55859e4640 1 -- 172.21.15.131:0/1916128743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5580070d00 msgr2=0x7f55800c2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.770+0000 7f55859e4640 1 --2- 172.21.15.131:0/1916128743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5580070d00 0x7f55800c2950 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f557800afd0 tx=0x7f5578020930 comp rx=0 tx=0).stop 2023-09-12T14:54:32.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.770+0000 7f55859e4640 1 -- 172.21.15.131:0/1916128743 shutdown_connections 2023-09-12T14:54:32.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.770+0000 7f55859e4640 1 --2- 172.21.15.131:0/1916128743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55800c2e90 0x7f55800c3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.770+0000 7f55859e4640 1 --2- 172.21.15.131:0/1916128743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5580070d00 0x7f55800c2950 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.770+0000 7f55859e4640 1 -- 172.21.15.131:0/1916128743 >> 172.21.15.131:0/1916128743 conn(0x7f55800ccb90 msgr2=0x7f55800ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:32.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.770+0000 7f55859e4640 1 -- 172.21.15.131:0/1916128743 shutdown_connections 2023-09-12T14:54:32.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.770+0000 7f55859e4640 1 -- 172.21.15.131:0/1916128743 wait complete. 2023-09-12T14:54:32.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.774+0000 7f55859e4640 1 Processor -- start 2023-09-12T14:54:32.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f55859e4640 1 -- start start 2023-09-12T14:54:32.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f55859e4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5580070d00 0x7f558016a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f55859e4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55800c2e90 0x7f5580168fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f55859e4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f558016b7c0 0x7f55800c5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f55859e4640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5580168b80 con 0x7f558016b7c0 2023-09-12T14:54:32.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f55859e4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55800827f0 con 0x7f5580070d00 2023-09-12T14:54:32.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f55859e4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55801681e0 con 0x7f55800c2e90 2023-09-12T14:54:32.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f55851e3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f558016b7c0 0x7f55800c5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f55849e2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5580070d00 0x7f558016a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f55851e3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f558016b7c0 0x7f55800c5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56182/0 (socket says 172.21.15.131:56182) 2023-09-12T14:54:32.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f55851e3640 1 -- 172.21.15.131:0/2002403441 learned_addr learned my addr 172.21.15.131:0/2002403441 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:32.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f55849e2640 1 -- 172.21.15.131:0/2002403441 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55800c2e90 msgr2=0x7f5580168fa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-12T14:54:32.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f55849e2640 1 --2- 172.21.15.131:0/2002403441 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55800c2e90 0x7f5580168fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f55849e2640 1 -- 172.21.15.131:0/2002403441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f558016b7c0 msgr2=0x7f55800c5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f55849e2640 1 --2- 172.21.15.131:0/2002403441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f558016b7c0 0x7f55800c5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f55849e2640 1 -- 172.21.15.131:0/2002403441 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55780083a0 con 0x7f5580070d00 2023-09-12T14:54:32.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f55849e2640 1 --2- 172.21.15.131:0/2002403441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5580070d00 0x7f558016a460 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f557400c550 tx=0x7f557400ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:32.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7fb22e181640 1 -- 172.21.15.131:0/3898334944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) v1 -- 0x7fb22804e110 con 0x7fb22815b9e0 2023-09-12T14:54:32.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7fb21dffb640 1 -- 172.21.15.131:0/3898334944 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v13)=0 v13) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb22001f060 con 0x7fb22815b9e0 2023-09-12T14:54:32.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.778+0000 7f5b957fa640 1 -- 172.21.15.131:0/3906470640 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f5b8c03bb20 con 0x7f5b980caab0 2023-09-12T14:54:32.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f4f1b7fe640 1 --2- 172.21.15.131:0/3476647534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f1c0c2960 0x7f4f1c164f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f3d96c41640 1 -- 172.21.15.131:0/2909785642 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d90151df0 con 0x7f3d900b0350 2023-09-12T14:54:32.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f3d96c41640 1 -- 172.21.15.131:0/2909785642 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d90143530 con 0x7f3d900b0350 2023-09-12T14:54:32.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f4f1bfff640 1 -- 172.21.15.131:0/3476647534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f1c0c2960 msgr2=0x7f4f1c164f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f4f1bfff640 1 --2- 172.21.15.131:0/3476647534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f1c0c2960 0x7f4f1c164f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f4f1bfff640 1 -- 172.21.15.131:0/3476647534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f1c156fb0 msgr2=0x7f4f1c161130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f4f1bfff640 1 --2- 172.21.15.131:0/3476647534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f1c156fb0 0x7f4f1c161130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f4f1bfff640 1 -- 172.21.15.131:0/3476647534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f140098e0 con 0x7f4f1c0708b0 2023-09-12T14:54:32.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f4f1b7fe640 1 --2- 172.21.15.131:0/3476647534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f1c0c2960 0x7f4f1c164f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:54:32.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.782+0000 7f19ebd11640 1 -- start start 2023-09-12T14:54:32.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f19ebd11640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19e40af8b0 0x7f19e414de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f19ebd11640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19e40afe70 0x7f19e4155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f19ebd11640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19e415e140 0x7f19e4200aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f19ebd11640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e4155ed0 con 0x7f19e40af8b0 2023-09-12T14:54:32.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f19ebd11640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e4156730 con 0x7f19e40afe70 2023-09-12T14:54:32.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f19ebd11640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e4156170 con 0x7f19e415e140 2023-09-12T14:54:32.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f4f1bfff640 1 --2- 172.21.15.131:0/3476647534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f1c0708b0 0x7f4f1c147650 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000b410 tx=0x7f4f1000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:32.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f19e92ac640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19e40afe70 0x7f19e4155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f19e92ac640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19e40afe70 0x7f19e4155990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:50924/0 (socket says 172.21.15.131:50924) 2023-09-12T14:54:32.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f19e92ac640 1 -- 172.21.15.131:0/4298498 learned_addr learned my addr 172.21.15.131:0/4298498 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:32.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f19ea2ae640 1 --2- 172.21.15.131:0/4298498 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19e415e140 0x7f19e4200aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f19e9aad640 1 --2- 172.21.15.131:0/4298498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19e40af8b0 0x7f19e414de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f3d8d7fa640 1 -- 172.21.15.131:0/2909785642 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3d7c00eeb0 con 0x7f3d900b0350 2023-09-12T14:54:32.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f3d8d7fa640 1 -- 172.21.15.131:0/2909785642 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c010670 con 0x7f3d900b0350 2023-09-12T14:54:32.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f19e92ac640 1 -- 172.21.15.131:0/4298498 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19e415e140 msgr2=0x7f19e4200aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f19e92ac640 1 --2- 172.21.15.131:0/4298498 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19e415e140 0x7f19e4200aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f19e92ac640 1 -- 172.21.15.131:0/4298498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19e40af8b0 msgr2=0x7f19e414de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f19e92ac640 1 --2- 172.21.15.131:0/4298498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19e40af8b0 0x7f19e414de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f19e92ac640 1 -- 172.21.15.131:0/4298498 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19dc009820 con 0x7f19e40afe70 2023-09-12T14:54:32.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f19e9aad640 1 --2- 172.21.15.131:0/4298498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19e40af8b0 0x7f19e414de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:54:32.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f19e92ac640 1 --2- 172.21.15.131:0/4298498 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19e40afe70 0x7f19e4155990 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f19e000c480 tx=0x7f19e000c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:32.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f557d7fa640 1 -- 172.21.15.131:0/2002403441 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5574005480 con 0x7f5580070d00 2023-09-12T14:54:32.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f55859e4640 1 -- 172.21.15.131:0/2002403441 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55801684c0 con 0x7f5580070d00 2023-09-12T14:54:32.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f55859e4640 1 -- 172.21.15.131:0/2002403441 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f558016ae80 con 0x7f5580070d00 2023-09-12T14:54:32.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f557d7fa640 1 -- 172.21.15.131:0/2002403441 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f55740036f0 con 0x7f5580070d00 2023-09-12T14:54:32.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f557d7fa640 1 -- 172.21.15.131:0/2002403441 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5574003b00 con 0x7f5580070d00 2023-09-12T14:54:32.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f3d96c41640 1 -- 172.21.15.131:0/2909785642 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d70005260 con 0x7f3d900b0350 2023-09-12T14:54:32.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7fb22e181640 1 -- 172.21.15.131:0/3898334944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb22815b9e0 msgr2=0x7fb2281f4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7fb22e181640 1 --2- 172.21.15.131:0/3898334944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb22815b9e0 0x7fb2281f4ea0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb22000e970 tx=0x7fb22000ee30 comp rx=0 tx=0).stop 2023-09-12T14:54:32.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7fb22e181640 1 -- 172.21.15.131:0/3898334944 shutdown_connections 2023-09-12T14:54:32.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7fb22e181640 1 --2- 172.21.15.131:0/3898334944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb22815b9e0 0x7fb2281f4ea0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7fb22e181640 1 --2- 172.21.15.131:0/3898334944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb22815b590 0x7fb2281f4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7fb22e181640 1 --2- 172.21.15.131:0/3898334944 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2280c8d00 0x7fb22814d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f4f18ff9640 1 -- 172.21.15.131:0/3476647534 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f10011830 con 0x7f4f1c0708b0 2023-09-12T14:54:32.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f4f21785640 1 -- 172.21.15.131:0/3476647534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f1c160930 con 0x7f4f1c0708b0 2023-09-12T14:54:32.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7f4f21785640 1 -- 172.21.15.131:0/3476647534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1c147b90 con 0x7f4f1c0708b0 2023-09-12T14:54:32.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.786+0000 7fb22e181640 1 -- 172.21.15.131:0/3898334944 >> 172.21.15.131:0/3898334944 conn(0x7fb2280ccb90 msgr2=0x7fb2280cb320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:32.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.790+0000 7f19da7fc640 1 -- 172.21.15.131:0/4298498 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0012830 con 0x7f19e40afe70 2023-09-12T14:54:32.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.790+0000 7f19ebd11640 1 -- 172.21.15.131:0/4298498 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e4156450 con 0x7f19e40afe70 2023-09-12T14:54:32.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.790+0000 7f19ebd11640 1 -- 172.21.15.131:0/4298498 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e415c7d0 con 0x7f19e40afe70 2023-09-12T14:54:32.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.790+0000 7f4f18ff9640 1 -- 172.21.15.131:0/3476647534 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4f10011a80 con 0x7f4f1c0708b0 2023-09-12T14:54:32.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.790+0000 7f4f18ff9640 1 -- 172.21.15.131:0/3476647534 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f10011e70 con 0x7f4f1c0708b0 2023-09-12T14:54:32.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.790+0000 7f4f18ff9640 1 -- 172.21.15.131:0/3476647534 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4f1001a700 con 0x7f4f1c0708b0 2023-09-12T14:54:32.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.790+0000 7f4f18ff9640 1 -- 172.21.15.131:0/3476647534 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4f10019970 con 0x7f4f1c0708b0 2023-09-12T14:54:32.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.790+0000 7f3d8d7fa640 1 -- 172.21.15.131:0/2909785642 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3d7c0047d0 con 0x7f3d900b0350 2023-09-12T14:54:32.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.790+0000 7f3d8d7fa640 1 -- 172.21.15.131:0/2909785642 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3d7c014070 con 0x7f3d900b0350 2023-09-12T14:54:32.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.790+0000 7fb22e181640 1 -- 172.21.15.131:0/3898334944 shutdown_connections 2023-09-12T14:54:32.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.790+0000 7fb22e181640 1 -- 172.21.15.131:0/3898334944 wait complete. 2023-09-12T14:54:32.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.790+0000 7f19da7fc640 1 -- 172.21.15.131:0/4298498 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f19e0012a80 con 0x7f19e40afe70 2023-09-12T14:54:32.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.790+0000 7f19da7fc640 1 -- 172.21.15.131:0/4298498 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0011380 con 0x7f19e40afe70 2023-09-12T14:54:32.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.790+0000 7f19da7fc640 1 -- 172.21.15.131:0/4298498 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f19e00118a0 con 0x7f19e40afe70 2023-09-12T14:54:32.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.790+0000 7f19da7fc640 1 -- 172.21.15.131:0/4298498 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f19e001b9a0 con 0x7f19e40afe70 2023-09-12T14:54:32.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.790+0000 7f19ebd11640 1 -- 172.21.15.131:0/4298498 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19cc005260 con 0x7f19e40afe70 2023-09-12T14:54:32.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.794+0000 7f557d7fa640 1 -- 172.21.15.131:0/2002403441 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f557401c440 con 0x7f5580070d00 2023-09-12T14:54:32.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.794+0000 7f557d7fa640 1 -- 172.21.15.131:0/2002403441 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5574012830 con 0x7f5580070d00 2023-09-12T14:54:32.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.794+0000 7f55859e4640 1 -- 172.21.15.131:0/2002403441 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5560005260 con 0x7f5580070d00 2023-09-12T14:54:32.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.794+0000 7f4f21785640 1 -- 172.21.15.131:0/3476647534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4efc005260 con 0x7f4f1c0708b0 2023-09-12T14:54:32.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.798+0000 7f19da7fc640 1 -- 172.21.15.131:0/4298498 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f19e0016020 con 0x7f19e40afe70 2023-09-12T14:54:32.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.794+0000 7f3d8d7fa640 1 -- 172.21.15.131:0/2909785642 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f3d7c005550 con 0x7f3d900b0350 2023-09-12T14:54:32.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.798+0000 7f4f18ff9640 1 -- 172.21.15.131:0/3476647534 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f4f10039b20 con 0x7f4f1c0708b0 2023-09-12T14:54:32.832 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi131') 2023-09-12T14:54:32.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.802+0000 7f557d7fa640 1 -- 172.21.15.131:0/2002403441 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f5574017070 con 0x7f5580070d00 2023-09-12T14:54:32.833 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi131') 2023-09-12T14:54:32.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.830+0000 7f8ac136f640 1 -- 172.21.15.131:0/3807134854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8abc0c9fe0 msgr2=0x7f8abc1566f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.830+0000 7f8ac136f640 1 --2- 172.21.15.131:0/3807134854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8abc0c9fe0 0x7f8abc1566f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0009870 tx=0x7f8ab001c260 comp rx=0 tx=0).stop 2023-09-12T14:54:32.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.830+0000 7f8ac136f640 1 -- 172.21.15.131:0/3807134854 shutdown_connections 2023-09-12T14:54:32.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.830+0000 7f8ac136f640 1 --2- 172.21.15.131:0/3807134854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8abc0c9fe0 0x7f8abc1566f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.830+0000 7f8ac136f640 1 -- 172.21.15.131:0/3807134854 >> 172.21.15.131:0/3807134854 conn(0x7f8abc0cdc20 msgr2=0x7f8abc0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:32.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.830+0000 7f8ac136f640 1 -- 172.21.15.131:0/3807134854 shutdown_connections 2023-09-12T14:54:32.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.830+0000 7f8ac136f640 1 -- 172.21.15.131:0/3807134854 wait complete. 2023-09-12T14:54:32.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.834+0000 7f8ac136f640 1 Processor -- start 2023-09-12T14:54:32.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.834+0000 7f8ac136f640 1 -- start start 2023-09-12T14:54:32.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.834+0000 7f8ac136f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8abc0694a0 0x7f8abc06c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.834+0000 7f8ac136f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8abc069880 0x7f8abc14d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.834+0000 7f8ac136f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8abc1494f0 0x7f8abc0b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:32.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.834+0000 7f8abbfff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8abc0694a0 0x7f8abc06c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.834+0000 7f8abbfff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8abc0694a0 0x7f8abc06c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:34020/0 (socket says 172.21.15.131:34020) 2023-09-12T14:54:32.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.834+0000 7f8abbfff640 1 -- 172.21.15.131:0/2777748267 learned_addr learned my addr 172.21.15.131:0/2777748267 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:32.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.834+0000 7f8abb7fe640 1 --2- 172.21.15.131:0/2777748267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8abc069880 0x7f8abc14d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.834+0000 7f8ac136f640 1 -- 172.21.15.131:0/2777748267 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc1549c0 con 0x7f8abc1494f0 2023-09-12T14:54:32.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.834+0000 7f8ac136f640 1 -- 172.21.15.131:0/2777748267 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc155540 con 0x7f8abc069880 2023-09-12T14:54:32.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.834+0000 7f8ac136f640 1 -- 172.21.15.131:0/2777748267 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc154c10 con 0x7f8abc0694a0 2023-09-12T14:54:32.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.834+0000 7f8ac0b6e640 1 --2- 172.21.15.131:0/2777748267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8abc1494f0 0x7f8abc0b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:32.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.834+0000 7f8abbfff640 1 -- 172.21.15.131:0/2777748267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8abc069880 msgr2=0x7f8abc14d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.834+0000 7f8abbfff640 1 --2- 172.21.15.131:0/2777748267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8abc069880 0x7f8abc14d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.834+0000 7f8abbfff640 1 -- 172.21.15.131:0/2777748267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8abc1494f0 msgr2=0x7f8abc0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.834+0000 7f8abbfff640 1 --2- 172.21.15.131:0/2777748267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8abc1494f0 0x7f8abc0b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.834+0000 7f8abbfff640 1 -- 172.21.15.131:0/2777748267 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab00094a0 con 0x7f8abc0694a0 2023-09-12T14:54:32.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.834+0000 7f8ac0b6e640 1 --2- 172.21.15.131:0/2777748267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8abc1494f0 0x7f8abc0b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:54:32.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.834+0000 7f8abbfff640 1 --2- 172.21.15.131:0/2777748267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8abc0694a0 0x7f8abc06c620 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8aac007de0 tx=0x7f8aac00a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:32.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.838+0000 7ff69bfff640 1 -- 172.21.15.131:0/3250183637 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) v1 -- 0x7ff67c0050d0 con 0x7ff69c0ab750 2023-09-12T14:54:32.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.838+0000 7f8ab8ff9640 1 -- 172.21.15.131:0/2777748267 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac0052d0 con 0x7f8abc0694a0 2023-09-12T14:54:32.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.838+0000 7f8ac136f640 1 -- 172.21.15.131:0/2777748267 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8abc154ef0 con 0x7f8abc0694a0 2023-09-12T14:54:32.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.838+0000 7f8ac136f640 1 -- 172.21.15.131:0/2777748267 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8abc06cb60 con 0x7f8abc0694a0 2023-09-12T14:54:32.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.838+0000 7f8ab8ff9640 1 -- 172.21.15.131:0/2777748267 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8aac00ae10 con 0x7f8abc0694a0 2023-09-12T14:54:32.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.838+0000 7fcab6231640 1 -- 172.21.15.131:0/2715769937 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) v1 -- 0x7fcab014c180 con 0x7fcab014eba0 2023-09-12T14:54:32.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.838+0000 7f8ab8ff9640 1 -- 172.21.15.131:0/2777748267 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac0072b0 con 0x7f8abc0694a0 2023-09-12T14:54:32.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.838+0000 7f8ab8ff9640 1 -- 172.21.15.131:0/2777748267 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8aac0067c0 con 0x7f8abc0694a0 2023-09-12T14:54:32.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.838+0000 7f8ab8ff9640 1 -- 172.21.15.131:0/2777748267 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8aac003d40 con 0x7f8abc0694a0 2023-09-12T14:54:32.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.842+0000 7f8ac136f640 1 -- 172.21.15.131:0/2777748267 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a9c005260 con 0x7f8abc0694a0 2023-09-12T14:54:32.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.846+0000 7ff683fff640 1 -- 172.21.15.131:0/3250183637 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v14)=0 v14) v1 ==== 115+0+0 (secure 0 0 0) 0x7ff6900b2070 con 0x7ff69c0ab750 2023-09-12T14:54:32.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.850+0000 7ff69bfff640 1 -- 172.21.15.131:0/3250183637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff69c0ab750 msgr2=0x7ff69c14d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.850+0000 7ff69bfff640 1 --2- 172.21.15.131:0/3250183637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff69c0ab750 0x7ff69c14d4c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff69001a510 tx=0x7ff6900a5780 comp rx=0 tx=0).stop 2023-09-12T14:54:32.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.850+0000 7ff69bfff640 1 -- 172.21.15.131:0/3250183637 shutdown_connections 2023-09-12T14:54:32.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.850+0000 7ff69bfff640 1 --2- 172.21.15.131:0/3250183637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff69c15eab0 0x7ff69c15a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.850+0000 7ff69bfff640 1 --2- 172.21.15.131:0/3250183637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff69c0cc740 0x7ff69c14da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.850+0000 7ff69bfff640 1 --2- 172.21.15.131:0/3250183637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff69c0ab750 0x7ff69c14d4c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.850+0000 7ff69bfff640 1 -- 172.21.15.131:0/3250183637 >> 172.21.15.131:0/3250183637 conn(0x7ff69c0ccb90 msgr2=0x7ff69c154540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:32.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.850+0000 7ff69bfff640 1 -- 172.21.15.131:0/3250183637 shutdown_connections 2023-09-12T14:54:32.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.850+0000 7ff69bfff640 1 -- 172.21.15.131:0/3250183637 wait complete. 2023-09-12T14:54:32.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.850+0000 7f8ab8ff9640 1 -- 172.21.15.131:0/2777748267 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f8aac022060 con 0x7f8abc0694a0 2023-09-12T14:54:32.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.854+0000 7fca97fff640 1 -- 172.21.15.131:0/2715769937 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v15)=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcaa80b6430 con 0x7fcab014eba0 2023-09-12T14:54:32.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.866+0000 7fcab6231640 1 -- 172.21.15.131:0/2715769937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcab014eba0 msgr2=0x7fcab014f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.866+0000 7fcab6231640 1 --2- 172.21.15.131:0/2715769937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcab014eba0 0x7fcab014f5d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80b0040 tx=0x7fcaa80045e0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.866+0000 7fcab6231640 1 -- 172.21.15.131:0/2715769937 shutdown_connections 2023-09-12T14:54:32.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.866+0000 7fcab6231640 1 --2- 172.21.15.131:0/2715769937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcab014eba0 0x7fcab014f5d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.866+0000 7fcab6231640 1 --2- 172.21.15.131:0/2715769937 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcab00cd250 0x7fcab014f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.866+0000 7fcab6231640 1 --2- 172.21.15.131:0/2715769937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcab00ccde0 0x7fcab0155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.866+0000 7fcab6231640 1 -- 172.21.15.131:0/2715769937 >> 172.21.15.131:0/2715769937 conn(0x7fcab0069640 msgr2=0x7fcab014b070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:32.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.866+0000 7fcab6231640 1 -- 172.21.15.131:0/2715769937 shutdown_connections 2023-09-12T14:54:32.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.866+0000 7fcab6231640 1 -- 172.21.15.131:0/2715769937 wait complete. 2023-09-12T14:54:32.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.886+0000 7fa940228640 1 -- 172.21.15.131:0/2309740765 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7fa9381f64b0 con 0x7fa9380caab0 2023-09-12T14:54:32.891 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi131') 2023-09-12T14:54:32.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.890+0000 7fa92e7fc640 1 -- 172.21.15.131:0/2309740765 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v16)=0 v16) v1 ==== 121+0+0 (secure 0 0 0) 0x7fa93402a020 con 0x7fa9380caab0 2023-09-12T14:54:32.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.890+0000 7fa940228640 1 -- 172.21.15.131:0/2309740765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9380caab0 msgr2=0x7fa938150010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.890+0000 7fa940228640 1 --2- 172.21.15.131:0/2309740765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9380caab0 0x7fa938150010 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa934009ec0 tx=0x7fa93400c430 comp rx=0 tx=0).stop 2023-09-12T14:54:32.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.894+0000 7fa940228640 1 -- 172.21.15.131:0/2309740765 shutdown_connections 2023-09-12T14:54:32.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.894+0000 7fa940228640 1 --2- 172.21.15.131:0/2309740765 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9380ccde0 0x7fa9380caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.894+0000 7fa940228640 1 --2- 172.21.15.131:0/2309740765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9380caab0 0x7fa938150010 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.894+0000 7fa940228640 1 --2- 172.21.15.131:0/2309740765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa93806ee20 0x7fa938144d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.894+0000 7fa940228640 1 -- 172.21.15.131:0/2309740765 >> 172.21.15.131:0/2309740765 conn(0x7fa938069640 msgr2=0x7fa938146160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:32.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.894+0000 7fa940228640 1 -- 172.21.15.131:0/2309740765 shutdown_connections 2023-09-12T14:54:32.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.894+0000 7fa940228640 1 -- 172.21.15.131:0/2309740765 wait complete. 2023-09-12T14:54:32.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.894+0000 7f4f21785640 1 -- 172.21.15.131:0/3476647534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7f4efc0050d0 con 0x7f4f1c0708b0 2023-09-12T14:54:32.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.898+0000 7f5b9edd5640 1 -- 172.21.15.131:0/3906470640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) v1 -- 0x7f5b9806fa50 con 0x7f5b980caab0 2023-09-12T14:54:32.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.902+0000 7f4f18ff9640 1 -- 172.21.15.131:0/3476647534 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v17)=0 v17) v1 ==== 121+0+0 (secure 0 0 0) 0x7f4f10028020 con 0x7f4f1c0708b0 2023-09-12T14:54:32.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.902+0000 7f55859e4640 1 -- 172.21.15.131:0/2002403441 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) v1 -- 0x7f55600050d0 con 0x7f5580070d00 2023-09-12T14:54:32.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.902+0000 7f4f21785640 1 -- 172.21.15.131:0/3476647534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f1c0708b0 msgr2=0x7f4f1c147650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.902+0000 7f4f21785640 1 --2- 172.21.15.131:0/3476647534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f1c0708b0 0x7f4f1c147650 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000b410 tx=0x7f4f1000b8d0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.902+0000 7f4f21785640 1 -- 172.21.15.131:0/3476647534 shutdown_connections 2023-09-12T14:54:32.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.902+0000 7f4f21785640 1 --2- 172.21.15.131:0/3476647534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f1c0c2960 0x7f4f1c164f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.902+0000 7f4f21785640 1 --2- 172.21.15.131:0/3476647534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f1c156fb0 0x7f4f1c161130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.902+0000 7f4f21785640 1 --2- 172.21.15.131:0/3476647534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f1c0708b0 0x7f4f1c147650 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.902+0000 7f4f21785640 1 -- 172.21.15.131:0/3476647534 >> 172.21.15.131:0/3476647534 conn(0x7f4f1c069240 msgr2=0x7f4f1c0caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:32.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.902+0000 7f4f21785640 1 -- 172.21.15.131:0/3476647534 shutdown_connections 2023-09-12T14:54:32.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.902+0000 7f4f21785640 1 -- 172.21.15.131:0/3476647534 wait complete. 2023-09-12T14:54:32.914 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi131') 2023-09-12T14:54:32.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.910+0000 7f3d96c41640 1 -- 172.21.15.131:0/2909785642 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) v1 -- 0x7f3d70005880 con 0x7f3d900b0350 2023-09-12T14:54:32.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.910+0000 7f5b957fa640 1 -- 172.21.15.131:0/3906470640 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v18)=0 v18) v1 ==== 121+0+0 (secure 0 0 0) 0x7f5b8c02a020 con 0x7f5b980caab0 2023-09-12T14:54:32.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.910+0000 7f5b7effd640 1 -- 172.21.15.131:0/3906470640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b980caab0 msgr2=0x7f5b980b0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.910+0000 7f5b7effd640 1 --2- 172.21.15.131:0/3906470640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b980caab0 0x7f5b980b0570 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00ee40 tx=0x7f5b8c00c430 comp rx=0 tx=0).stop 2023-09-12T14:54:32.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.910+0000 7f5b7effd640 1 -- 172.21.15.131:0/3906470640 shutdown_connections 2023-09-12T14:54:32.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.910+0000 7f5b7effd640 1 --2- 172.21.15.131:0/3906470640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b980ccde0 0x7f5b980adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.910+0000 7f5b7effd640 1 --2- 172.21.15.131:0/3906470640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b980caab0 0x7f5b980b0570 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.910+0000 7f5b7effd640 1 --2- 172.21.15.131:0/3906470640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b9806ee20 0x7f5b9813f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.910+0000 7f5b7effd640 1 -- 172.21.15.131:0/3906470640 >> 172.21.15.131:0/3906470640 conn(0x7f5b98069640 msgr2=0x7f5b9815e860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:32.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.910+0000 7f5b7effd640 1 -- 172.21.15.131:0/3906470640 shutdown_connections 2023-09-12T14:54:32.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.910+0000 7f5b7effd640 1 -- 172.21.15.131:0/3906470640 wait complete. 2023-09-12T14:54:32.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.918+0000 7f19ebd11640 1 -- 172.21.15.131:0/4298498 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) v1 -- 0x7f19cc0050d0 con 0x7f19e40afe70 2023-09-12T14:54:32.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.922+0000 7f557d7fa640 1 -- 172.21.15.131:0/2002403441 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v19)=0 v19) v1 ==== 119+0+0 (secure 0 0 0) 0x7f5574012a40 con 0x7f5580070d00 2023-09-12T14:54:32.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.926+0000 7f55859e4640 1 -- 172.21.15.131:0/2002403441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5580070d00 msgr2=0x7f558016a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.926+0000 7f55859e4640 1 --2- 172.21.15.131:0/2002403441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5580070d00 0x7f558016a460 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f557400c550 tx=0x7f557400ca10 comp rx=0 tx=0).stop 2023-09-12T14:54:32.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.926+0000 7f55859e4640 1 -- 172.21.15.131:0/2002403441 shutdown_connections 2023-09-12T14:54:32.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.926+0000 7f55859e4640 1 --2- 172.21.15.131:0/2002403441 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55800c2e90 0x7f5580168fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.926+0000 7f55859e4640 1 --2- 172.21.15.131:0/2002403441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f558016b7c0 0x7f55800c5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.926+0000 7f55859e4640 1 --2- 172.21.15.131:0/2002403441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5580070d00 0x7f558016a460 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.926+0000 7f55859e4640 1 -- 172.21.15.131:0/2002403441 >> 172.21.15.131:0/2002403441 conn(0x7f55800ccb90 msgr2=0x7f558006c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:32.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.926+0000 7f55859e4640 1 -- 172.21.15.131:0/2002403441 shutdown_connections 2023-09-12T14:54:32.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.926+0000 7f55859e4640 1 -- 172.21.15.131:0/2002403441 wait complete. 2023-09-12T14:54:32.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.930+0000 7f3d8d7fa640 1 -- 172.21.15.131:0/2909785642 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v20)=0 v20) v1 ==== 120+0+0 (secure 0 0 0) 0x7f3d7c004a80 con 0x7f3d900b0350 2023-09-12T14:54:32.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.930+0000 7f3d96c41640 1 -- 172.21.15.131:0/2909785642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d900b0350 msgr2=0x7f3d901f7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.930+0000 7f3d96c41640 1 --2- 172.21.15.131:0/2909785642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d900b0350 0x7f3d901f7980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c00e7a0 tx=0x7f3d7c00ec60 comp rx=0 tx=0).stop 2023-09-12T14:54:32.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.930+0000 7f3d96c41640 1 -- 172.21.15.131:0/2909785642 shutdown_connections 2023-09-12T14:54:32.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.930+0000 7f3d96c41640 1 --2- 172.21.15.131:0/2909785642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d9015bc30 0x7f3d901515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.930+0000 7f3d96c41640 1 --2- 172.21.15.131:0/2909785642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d90158c70 0x7f3d902017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.930+0000 7f3d96c41640 1 --2- 172.21.15.131:0/2909785642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d900b0350 0x7f3d901f7980 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.930+0000 7f3d96c41640 1 -- 172.21.15.131:0/2909785642 >> 172.21.15.131:0/2909785642 conn(0x7f3d90069640 msgr2=0x7f3d9014b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:32.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.930+0000 7f3d96c41640 1 -- 172.21.15.131:0/2909785642 shutdown_connections 2023-09-12T14:54:32.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.930+0000 7f3d96c41640 1 -- 172.21.15.131:0/2909785642 wait complete. 2023-09-12T14:54:32.938 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi131') 2023-09-12T14:54:32.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.934+0000 7f19da7fc640 1 -- 172.21.15.131:0/4298498 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v21)=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7f19e001b400 con 0x7f19e40afe70 2023-09-12T14:54:32.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.934+0000 7f19ebd11640 1 -- 172.21.15.131:0/4298498 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19e40afe70 msgr2=0x7f19e4155990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.934+0000 7f19ebd11640 1 --2- 172.21.15.131:0/4298498 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19e40afe70 0x7f19e4155990 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f19e000c480 tx=0x7f19e000c940 comp rx=0 tx=0).stop 2023-09-12T14:54:32.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.934+0000 7f19ebd11640 1 -- 172.21.15.131:0/4298498 shutdown_connections 2023-09-12T14:54:32.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.934+0000 7f19ebd11640 1 --2- 172.21.15.131:0/4298498 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19e415e140 0x7f19e4200aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.934+0000 7f19ebd11640 1 --2- 172.21.15.131:0/4298498 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19e40afe70 0x7f19e4155990 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.934+0000 7f19ebd11640 1 --2- 172.21.15.131:0/4298498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19e40af8b0 0x7f19e414de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.934+0000 7f19ebd11640 1 -- 172.21.15.131:0/4298498 >> 172.21.15.131:0/4298498 conn(0x7f19e40cd4d0 msgr2=0x7f19e40b1930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:32.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.934+0000 7f19ebd11640 1 -- 172.21.15.131:0/4298498 shutdown_connections 2023-09-12T14:54:32.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.934+0000 7f19ebd11640 1 -- 172.21.15.131:0/4298498 wait complete. 2023-09-12T14:54:32.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.950+0000 7f8ac136f640 1 -- 172.21.15.131:0/2777748267 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) v1 -- 0x7f8a9c005820 con 0x7f8abc0694a0 2023-09-12T14:54:32.957 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi131') 2023-09-12T14:54:32.957 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi131') 2023-09-12T14:54:32.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.958+0000 7f8ab8ff9640 1 -- 172.21.15.131:0/2777748267 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v22)=0 v22) v1 ==== 120+0+0 (secure 0 0 0) 0x7f8aac018070 con 0x7f8abc0694a0 2023-09-12T14:54:32.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.958+0000 7f8ac136f640 1 -- 172.21.15.131:0/2777748267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8abc0694a0 msgr2=0x7f8abc06c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:32.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.958+0000 7f8ac136f640 1 --2- 172.21.15.131:0/2777748267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8abc0694a0 0x7f8abc06c620 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8aac007de0 tx=0x7f8aac00a430 comp rx=0 tx=0).stop 2023-09-12T14:54:32.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.958+0000 7f8ac136f640 1 -- 172.21.15.131:0/2777748267 shutdown_connections 2023-09-12T14:54:32.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.958+0000 7f8ac136f640 1 --2- 172.21.15.131:0/2777748267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8abc1494f0 0x7f8abc0b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.958+0000 7f8ac136f640 1 --2- 172.21.15.131:0/2777748267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8abc069880 0x7f8abc14d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.958+0000 7f8ac136f640 1 --2- 172.21.15.131:0/2777748267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8abc0694a0 0x7f8abc06c620 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:32.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.958+0000 7f8ac136f640 1 -- 172.21.15.131:0/2777748267 >> 172.21.15.131:0/2777748267 conn(0x7f8abc0cdc20 msgr2=0x7f8abc169620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:32.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.958+0000 7f8ac136f640 1 -- 172.21.15.131:0/2777748267 shutdown_connections 2023-09-12T14:54:32.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:32.958+0000 7f8ac136f640 1 -- 172.21.15.131:0/2777748267 wait complete. 2023-09-12T14:54:32.978 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi131') 2023-09-12T14:54:32.978 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi131') 2023-09-12T14:54:32.979 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi131') 2023-09-12T14:54:32.979 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-09-12T14:54:32.979 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-09-12T14:54:32.980 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-09-12T14:54:33.022 INFO:tasks.ceph.mgr.y:Started 2023-09-12T14:54:33.022 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-09-12T14:54:33.023 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-09-12T14:54:33.026 INFO:tasks.ceph.mgr.x:Started 2023-09-12T14:54:33.026 INFO:tasks.ceph:Setting crush tunables to default 2023-09-12T14:54:33.027 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph osd crush tunables default 2023-09-12T14:54:33.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.126+0000 7f63aa1df640 1 -- 172.21.15.131:0/2204871572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63a40b1650 msgr2=0x7f63a4159270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:33.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.126+0000 7f63aa1df640 1 --2- 172.21.15.131:0/2204871572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63a40b1650 0x7f63a4159270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f63a4070e80 tx=0x7f639c01cb30 comp rx=0 tx=0).stop 2023-09-12T14:54:33.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.126+0000 7f63aa1df640 1 -- 172.21.15.131:0/2204871572 shutdown_connections 2023-09-12T14:54:33.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.126+0000 7f63aa1df640 1 --2- 172.21.15.131:0/2204871572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63a40b1650 0x7f63a4159270 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:33.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.126+0000 7f63aa1df640 1 --2- 172.21.15.131:0/2204871572 >> v2:172.21.15.187:3300/0 conn(0x7f63a40701c0 0x7f63a40705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:33.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.126+0000 7f63aa1df640 1 -- 172.21.15.131:0/2204871572 >> 172.21.15.131:0/2204871572 conn(0x7f63a40afab0 msgr2=0x7f63a40afec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:33.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.126+0000 7f63aa1df640 1 -- 172.21.15.131:0/2204871572 shutdown_connections 2023-09-12T14:54:33.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.126+0000 7f63aa1df640 1 -- 172.21.15.131:0/2204871572 wait complete. 2023-09-12T14:54:33.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f63aa1df640 1 Processor -- start 2023-09-12T14:54:33.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f63aa1df640 1 -- start start 2023-09-12T14:54:33.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f63aa1df640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63a40701c0 0x7f63a415b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:33.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f63aa1df640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63a40b1650 0x7f63a4147f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:33.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f63aa1df640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63a40b22f0 0x7f63a4148440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:33.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f63aa1df640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a4149490 con 0x7f63a40b1650 2023-09-12T14:54:33.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f63aa1df640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a414a010 con 0x7f63a40701c0 2023-09-12T14:54:33.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f63aa1df640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a4149670 con 0x7f63a40b22f0 2023-09-12T14:54:33.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f639bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63a40b1650 0x7f63a4147f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:33.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f639bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63a40b1650 0x7f63a4147f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56268/0 (socket says 172.21.15.131:56268) 2023-09-12T14:54:33.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f639bfff640 1 -- 172.21.15.131:0/2680211761 learned_addr learned my addr 172.21.15.131:0/2680211761 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:33.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f63a37fe640 1 --2- 172.21.15.131:0/2680211761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63a40701c0 0x7f63a415b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:33.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f63a3fff640 1 --2- 172.21.15.131:0/2680211761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63a40b22f0 0x7f63a4148440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:33.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f639bfff640 1 -- 172.21.15.131:0/2680211761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63a40b22f0 msgr2=0x7f63a4148440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:33.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f639bfff640 1 --2- 172.21.15.131:0/2680211761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63a40b22f0 0x7f63a4148440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:33.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f639bfff640 1 -- 172.21.15.131:0/2680211761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63a40701c0 msgr2=0x7f63a415b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:33.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f639bfff640 1 --2- 172.21.15.131:0/2680211761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63a40701c0 0x7f63a415b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:33.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f639bfff640 1 -- 172.21.15.131:0/2680211761 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639c009820 con 0x7f63a40b1650 2023-09-12T14:54:33.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f63a37fe640 1 --2- 172.21.15.131:0/2680211761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63a40701c0 0x7f63a415b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:54:33.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f639bfff640 1 --2- 172.21.15.131:0/2680211761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63a40b1650 0x7f63a4147f00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f638c00c420 tx=0x7f638c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:33.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f63a0ff9640 1 -- 172.21.15.131:0/2680211761 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c0054e0 con 0x7f63a40b1650 2023-09-12T14:54:33.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f63a0ff9640 1 -- 172.21.15.131:0/2680211761 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f638c003600 con 0x7f63a40b1650 2023-09-12T14:54:33.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f63aa1df640 1 -- 172.21.15.131:0/2680211761 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a4149950 con 0x7f63a40b1650 2023-09-12T14:54:33.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f63a0ff9640 1 -- 172.21.15.131:0/2680211761 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c0039f0 con 0x7f63a40b1650 2023-09-12T14:54:33.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f63aa1df640 1 -- 172.21.15.131:0/2680211761 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a4155240 con 0x7f63a40b1650 2023-09-12T14:54:33.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f63a0ff9640 1 -- 172.21.15.131:0/2680211761 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f638c01c440 con 0x7f63a40b1650 2023-09-12T14:54:33.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f63a0ff9640 1 -- 172.21.15.131:0/2680211761 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f638c012850 con 0x7f63a40b1650 2023-09-12T14:54:33.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.130+0000 7f63aa1df640 1 -- 172.21.15.131:0/2680211761 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a414a150 con 0x7f63a40b1650 2023-09-12T14:54:33.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.134+0000 7f63a0ff9640 1 -- 172.21.15.131:0/2680211761 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f63a414a150 con 0x7f63a40b1650 2023-09-12T14:54:33.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.190+0000 7f63aa1df640 1 -- 172.21.15.131:0/2680211761 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f63a4073580 con 0x7f63a40b1650 2023-09-12T14:54:33.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.194+0000 7f63a0ff9640 1 -- 172.21.15.131:0/2680211761 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f638c017070 con 0x7f63a40b1650 2023-09-12T14:54:33.198 INFO:teuthology.orchestra.run.smithi131.stderr:adjusted tunables profile to default 2023-09-12T14:54:33.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.194+0000 7f63aa1df640 1 -- 172.21.15.131:0/2680211761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63a40b1650 msgr2=0x7f63a4147f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:33.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.194+0000 7f63aa1df640 1 --2- 172.21.15.131:0/2680211761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63a40b1650 0x7f63a4147f00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f638c00c420 tx=0x7f638c00c8e0 comp rx=0 tx=0).stop 2023-09-12T14:54:33.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.194+0000 7f63aa1df640 1 -- 172.21.15.131:0/2680211761 shutdown_connections 2023-09-12T14:54:33.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.194+0000 7f63aa1df640 1 --2- 172.21.15.131:0/2680211761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63a40b22f0 0x7f63a4148440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:33.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.194+0000 7f63aa1df640 1 --2- 172.21.15.131:0/2680211761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63a40b1650 0x7f63a4147f00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:33.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.194+0000 7f63aa1df640 1 --2- 172.21.15.131:0/2680211761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63a40701c0 0x7f63a415b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:33.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.194+0000 7f63aa1df640 1 -- 172.21.15.131:0/2680211761 >> 172.21.15.131:0/2680211761 conn(0x7f63a40afab0 msgr2=0x7f63a41567b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:33.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.194+0000 7f63aa1df640 1 -- 172.21.15.131:0/2680211761 shutdown_connections 2023-09-12T14:54:33.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:33.194+0000 7f63aa1df640 1 -- 172.21.15.131:0/2680211761 wait complete. 2023-09-12T14:54:33.213 INFO:tasks.ceph:check_enable_crimson: False 2023-09-12T14:54:33.214 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-09-12T14:54:33.214 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:54:33.215 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-09-12T14:54:33.224 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:54:33.224 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-09-12T14:54:33.273 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:54:33.274 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-09-12T14:54:33.298 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-09-12T14:54:33.294+0000 7f19e66bb180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-12T14:54:33.309 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:54:33.309 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-09-12T14:54:33.332 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-09-12T14:54:33.330+0000 7f0f107e2180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-12T14:54:33.364 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:54:33.365 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-09-12T14:54:33.378 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:54:33.379 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-09-12T14:54:33.431 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:54:33.431 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-09-12T14:54:33.486 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:54:33.487 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-09-12T14:54:33.538 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 2ddb30e8-fd9b-46c1-a667-0eaba00cdc18 0 2023-09-12T14:54:33.705 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.702+0000 7f9881231640 1 -- 172.21.15.187:0/1673251809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f987c0cf700 msgr2=0x7f987c0cbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:33.705 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.702+0000 7f9881231640 1 --2- 172.21.15.187:0/1673251809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f987c0cf700 0x7f987c0cbb10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f987c0ceb40 tx=0x7f987001c7a0 comp rx=0 tx=0).stop 2023-09-12T14:54:33.706 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.702+0000 7f9881231640 1 -- 172.21.15.187:0/1673251809 shutdown_connections 2023-09-12T14:54:33.706 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.702+0000 7f9881231640 1 --2- 172.21.15.187:0/1673251809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f987c0cf700 0x7f987c0cbb10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:33.706 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.702+0000 7f9881231640 1 --2- 172.21.15.187:0/1673251809 >> v2:172.21.15.131:3300/0 conn(0x7f987c15d570 0x7f987c0cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:33.706 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.702+0000 7f9881231640 1 -- 172.21.15.187:0/1673251809 >> 172.21.15.187:0/1673251809 conn(0x7f987c0cebe0 msgr2=0x7f987c0ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:33.706 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.702+0000 7f9881231640 1 -- 172.21.15.187:0/1673251809 shutdown_connections 2023-09-12T14:54:33.707 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.702+0000 7f9881231640 1 -- 172.21.15.187:0/1673251809 wait complete. 2023-09-12T14:54:33.709 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.702+0000 7f9881231640 1 Processor -- start 2023-09-12T14:54:33.709 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.702+0000 7f9881231640 1 -- start start 2023-09-12T14:54:33.709 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.702+0000 7f9881231640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f987c0b1ff0 0x7f987c1632c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:33.710 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.702+0000 7f9881231640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f987c0cf700 0x7f987c1f8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:33.710 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.702+0000 7f9881231640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f987c15d570 0x7f987c06abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:33.710 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.702+0000 7f9881231640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987c1f8b60 con 0x7f987c15d570 2023-09-12T14:54:33.710 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.702+0000 7f9881231640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987c1f8e40 con 0x7f987c0b1ff0 2023-09-12T14:54:33.711 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.702+0000 7f9881231640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f987c1f9120 con 0x7f987c0cf700 2023-09-12T14:54:33.711 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.702+0000 7f987ad9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f987c0b1ff0 0x7f987c1632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:33.711 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.702+0000 7f987a59c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f987c0cf700 0x7f987c1f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:33.711 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.702+0000 7f987ad9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f987c0b1ff0 0x7f987c1632c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:48660/0 (socket says 172.21.15.187:48660) 2023-09-12T14:54:33.712 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.702+0000 7f987ad9d640 1 -- 172.21.15.187:0/3643880281 learned_addr learned my addr 172.21.15.187:0/3643880281 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-09-12T14:54:33.712 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.706+0000 7f987b59e640 1 --2- 172.21.15.187:0/3643880281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f987c15d570 0x7f987c06abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:33.712 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.706+0000 7f987ad9d640 1 -- 172.21.15.187:0/3643880281 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f987c0cf700 msgr2=0x7f987c1f8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:33.713 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.706+0000 7f987ad9d640 1 --2- 172.21.15.187:0/3643880281 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f987c0cf700 0x7f987c1f8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:33.713 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.706+0000 7f987ad9d640 1 -- 172.21.15.187:0/3643880281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f987c15d570 msgr2=0x7f987c06abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:33.713 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.706+0000 7f987ad9d640 1 --2- 172.21.15.187:0/3643880281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f987c15d570 0x7f987c06abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:33.713 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.706+0000 7f987ad9d640 1 -- 172.21.15.187:0/3643880281 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9870009560 con 0x7f987c0b1ff0 2023-09-12T14:54:33.713 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.706+0000 7f987b59e640 1 --2- 172.21.15.187:0/3643880281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f987c15d570 0x7f987c06abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:54:33.714 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.706+0000 7f987ad9d640 1 --2- 172.21.15.187:0/3643880281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f987c0b1ff0 0x7f987c1632c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f986c00c970 tx=0x7f986c00ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:33.714 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.706+0000 7f98637fe640 1 -- 172.21.15.187:0/3643880281 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c007bb0 con 0x7f987c0b1ff0 2023-09-12T14:54:33.714 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.706+0000 7f98637fe640 1 -- 172.21.15.187:0/3643880281 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f986c007e00 con 0x7f987c0b1ff0 2023-09-12T14:54:33.714 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.706+0000 7f9881231640 1 -- 172.21.15.187:0/3643880281 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f987c06a3f0 con 0x7f987c0b1ff0 2023-09-12T14:54:33.714 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.706+0000 7f98637fe640 1 -- 172.21.15.187:0/3643880281 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c005290 con 0x7f987c0b1ff0 2023-09-12T14:54:33.715 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.706+0000 7f9881231640 1 -- 172.21.15.187:0/3643880281 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f987c163800 con 0x7f987c0b1ff0 2023-09-12T14:54:33.715 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.706+0000 7f98637fe640 1 -- 172.21.15.187:0/3643880281 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f986c0053f0 con 0x7f987c0b1ff0 2023-09-12T14:54:33.715 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.706+0000 7f98637fe640 1 -- 172.21.15.187:0/3643880281 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f986c00f820 con 0x7f987c0b1ff0 2023-09-12T14:54:33.715 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.706+0000 7f9881231640 1 -- 172.21.15.187:0/3643880281 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f987c073580 con 0x7f987c0b1ff0 2023-09-12T14:54:33.716 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.706+0000 7f98637fe640 1 -- 172.21.15.187:0/3643880281 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f986c01d060 con 0x7f987c0b1ff0 2023-09-12T14:54:33.779 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.774+0000 7f9881231640 1 -- 172.21.15.187:0/3643880281 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2ddb30e8-fd9b-46c1-a667-0eaba00cdc18", "id": 0} v 0) v1 -- 0x7f987c06a8a0 con 0x7f987c0b1ff0 2023-09-12T14:54:33.786 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.782+0000 7f98637fe640 1 -- 172.21.15.187:0/3643880281 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2ddb30e8-fd9b-46c1-a667-0eaba00cdc18", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f986c0270b0 con 0x7f987c0b1ff0 2023-09-12T14:54:33.786 INFO:teuthology.orchestra.run.smithi187.stdout:0 2023-09-12T14:54:33.787 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.782+0000 7f9881231640 1 -- 172.21.15.187:0/3643880281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f987c0b1ff0 msgr2=0x7f987c1632c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:33.788 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.782+0000 7f9881231640 1 --2- 172.21.15.187:0/3643880281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f987c0b1ff0 0x7f987c1632c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f986c00c970 tx=0x7f986c00ce30 comp rx=0 tx=0).stop 2023-09-12T14:54:33.788 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.782+0000 7f9881231640 1 -- 172.21.15.187:0/3643880281 shutdown_connections 2023-09-12T14:54:33.788 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.782+0000 7f9881231640 1 --2- 172.21.15.187:0/3643880281 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f987c0cf700 0x7f987c1f8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:33.788 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.782+0000 7f9881231640 1 --2- 172.21.15.187:0/3643880281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f987c15d570 0x7f987c06abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:33.788 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.786+0000 7f9881231640 1 --2- 172.21.15.187:0/3643880281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f987c0b1ff0 0x7f987c1632c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:33.789 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.786+0000 7f9881231640 1 -- 172.21.15.187:0/3643880281 >> 172.21.15.187:0/3643880281 conn(0x7f987c0cebe0 msgr2=0x7f987c06e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:33.789 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.786+0000 7f9881231640 1 -- 172.21.15.187:0/3643880281 shutdown_connections 2023-09-12T14:54:33.789 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.786+0000 7f9881231640 1 -- 172.21.15.187:0/3643880281 wait complete. 2023-09-12T14:54:33.802 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 095cca20-beb2-4d55-b8f8-f1910692ac5f 1 2023-09-12T14:54:33.829 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-09-12T14:54:33.826+0000 7f0f107e2180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-12T14:54:33.844 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-09-12T14:54:33.838+0000 7f19e66bb180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-12T14:54:33.921 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae808ca640 1 -- 172.21.15.187:0/3195979303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae7c157b10 msgr2=0x7fae7c0cc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:33.921 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae808ca640 1 --2- 172.21.15.187:0/3195979303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae7c157b10 0x7fae7c0cc0e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0ce120 tx=0x7fae6401c7e0 comp rx=0 tx=0).stop 2023-09-12T14:54:33.921 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae808ca640 1 -- 172.21.15.187:0/3195979303 shutdown_connections 2023-09-12T14:54:33.921 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae808ca640 1 --2- 172.21.15.187:0/3195979303 >> v2:172.21.15.131:3300/0 conn(0x7fae7c15b7c0 0x7fae7c15bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:33.922 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae808ca640 1 --2- 172.21.15.187:0/3195979303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae7c157b10 0x7fae7c0cc0e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:33.922 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae808ca640 1 -- 172.21.15.187:0/3195979303 >> 172.21.15.187:0/3195979303 conn(0x7fae7c06b310 msgr2=0x7fae7c06b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:33.922 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae808ca640 1 -- 172.21.15.187:0/3195979303 shutdown_connections 2023-09-12T14:54:33.922 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae808ca640 1 -- 172.21.15.187:0/3195979303 wait complete. 2023-09-12T14:54:33.924 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae808ca640 1 Processor -- start 2023-09-12T14:54:33.925 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae808ca640 1 -- start start 2023-09-12T14:54:33.925 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae808ca640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae7c06fa70 0x7fae7c1464d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:33.925 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae808ca640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae7c157b10 0x7fae7c1622d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:33.925 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae808ca640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae7c15b7c0 0x7fae7c0c2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:33.925 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae808ca640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c146a10 con 0x7fae7c157b10 2023-09-12T14:54:33.926 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae808ca640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c1528d0 con 0x7fae7c15b7c0 2023-09-12T14:54:33.926 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae808ca640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c15a430 con 0x7fae7c06fa70 2023-09-12T14:54:33.926 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae7ad9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae7c15b7c0 0x7fae7c0c2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:33.926 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae7ad9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae7c15b7c0 0x7fae7c0c2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:48674/0 (socket says 172.21.15.187:48674) 2023-09-12T14:54:33.926 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae7ad9d640 1 -- 172.21.15.187:0/483915040 learned_addr learned my addr 172.21.15.187:0/483915040 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-09-12T14:54:33.927 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae7a59c640 1 --2- 172.21.15.187:0/483915040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae7c06fa70 0x7fae7c1464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:33.927 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae79d9b640 1 --2- 172.21.15.187:0/483915040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae7c157b10 0x7fae7c1622d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:33.927 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae7ad9d640 1 -- 172.21.15.187:0/483915040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae7c06fa70 msgr2=0x7fae7c1464d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:33.927 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae7ad9d640 1 --2- 172.21.15.187:0/483915040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae7c06fa70 0x7fae7c1464d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:33.927 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae7ad9d640 1 -- 172.21.15.187:0/483915040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae7c157b10 msgr2=0x7fae7c1622d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:33.928 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae7ad9d640 1 --2- 172.21.15.187:0/483915040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae7c157b10 0x7fae7c1622d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:33.928 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae7ad9d640 1 -- 172.21.15.187:0/483915040 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae64009560 con 0x7fae7c15b7c0 2023-09-12T14:54:33.928 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae7ad9d640 1 --2- 172.21.15.187:0/483915040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae7c15b7c0 0x7fae7c0c2b00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fae7400cce0 tx=0x7fae7400a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:33.928 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae72ffd640 1 -- 172.21.15.187:0/483915040 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7400f450 con 0x7fae7c15b7c0 2023-09-12T14:54:33.928 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.918+0000 7fae72ffd640 1 -- 172.21.15.187:0/483915040 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fae7400f6c0 con 0x7fae7c15b7c0 2023-09-12T14:54:33.929 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.922+0000 7fae808ca640 1 -- 172.21.15.187:0/483915040 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae7c15a710 con 0x7fae7c15b7c0 2023-09-12T14:54:33.929 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.922+0000 7fae72ffd640 1 -- 172.21.15.187:0/483915040 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7400fad0 con 0x7fae7c15b7c0 2023-09-12T14:54:33.929 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.922+0000 7fae808ca640 1 -- 172.21.15.187:0/483915040 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7c15acd0 con 0x7fae7c15b7c0 2023-09-12T14:54:33.929 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.922+0000 7fae72ffd640 1 -- 172.21.15.187:0/483915040 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fae7400fc30 con 0x7fae7c15b7c0 2023-09-12T14:54:33.929 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.922+0000 7fae72ffd640 1 -- 172.21.15.187:0/483915040 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fae740217a0 con 0x7fae7c15b7c0 2023-09-12T14:54:33.930 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.922+0000 7fae808ca640 1 -- 172.21.15.187:0/483915040 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae7c06da50 con 0x7fae7c15b7c0 2023-09-12T14:54:33.930 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.922+0000 7fae72ffd640 1 -- 172.21.15.187:0/483915040 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fae74018cc0 con 0x7fae7c15b7c0 2023-09-12T14:54:33.993 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.990+0000 7fae808ca640 1 -- 172.21.15.187:0/483915040 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "095cca20-beb2-4d55-b8f8-f1910692ac5f", "id": 1} v 0) v1 -- 0x7fae7c073580 con 0x7fae7c15b7c0 2023-09-12T14:54:34.001 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.998+0000 7fae72ffd640 1 -- 172.21.15.187:0/483915040 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "095cca20-beb2-4d55-b8f8-f1910692ac5f", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fae74004b30 con 0x7fae7c15b7c0 2023-09-12T14:54:34.002 INFO:teuthology.orchestra.run.smithi187.stdout:1 2023-09-12T14:54:34.003 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.998+0000 7fae808ca640 1 -- 172.21.15.187:0/483915040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae7c15b7c0 msgr2=0x7fae7c0c2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:34.003 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.998+0000 7fae808ca640 1 --2- 172.21.15.187:0/483915040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae7c15b7c0 0x7fae7c0c2b00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fae7400cce0 tx=0x7fae7400a620 comp rx=0 tx=0).stop 2023-09-12T14:54:34.003 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.998+0000 7fae808ca640 1 -- 172.21.15.187:0/483915040 shutdown_connections 2023-09-12T14:54:34.003 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.998+0000 7fae808ca640 1 --2- 172.21.15.187:0/483915040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae7c15b7c0 0x7fae7c0c2b00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.004 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.998+0000 7fae808ca640 1 --2- 172.21.15.187:0/483915040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae7c157b10 0x7fae7c1622d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.004 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.998+0000 7fae808ca640 1 --2- 172.21.15.187:0/483915040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae7c06fa70 0x7fae7c1464d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.004 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.998+0000 7fae808ca640 1 -- 172.21.15.187:0/483915040 >> 172.21.15.187:0/483915040 conn(0x7fae7c06b310 msgr2=0x7fae7c153080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:34.004 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.998+0000 7fae808ca640 1 -- 172.21.15.187:0/483915040 shutdown_connections 2023-09-12T14:54:34.004 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:33.998+0000 7fae808ca640 1 -- 172.21.15.187:0/483915040 wait complete. 2023-09-12T14:54:34.015 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-09-12T14:54:34.010+0000 7f0f107e2180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-12T14:54:34.015 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-09-12T14:54:34.010+0000 7f19e66bb180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-12T14:54:34.018 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 14ba94e7-71fd-48c3-a92e-04b6274992fe 2 2023-09-12T14:54:34.180 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f8f54640 1 -- 172.21.15.187:0/2690677328 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04f406fdd0 msgr2=0x7f04f4070240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:34.181 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f8f54640 1 --2- 172.21.15.187:0/2690677328 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04f406fdd0 0x7f04f4070240 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f04ec009bc0 tx=0x7f04ec01cb50 comp rx=0 tx=0).stop 2023-09-12T14:54:34.181 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f8f54640 1 -- 172.21.15.187:0/2690677328 shutdown_connections 2023-09-12T14:54:34.181 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f8f54640 1 --2- 172.21.15.187:0/2690677328 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04f406fdd0 0x7f04f4070240 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.181 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f8f54640 1 --2- 172.21.15.187:0/2690677328 >> v2:172.21.15.187:3300/0 conn(0x7f04f41556e0 0x7f04f4158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.181 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f8f54640 1 -- 172.21.15.187:0/2690677328 >> 172.21.15.187:0/2690677328 conn(0x7f04f406a950 msgr2=0x7f04f40af5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:34.182 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f8f54640 1 -- 172.21.15.187:0/2690677328 shutdown_connections 2023-09-12T14:54:34.182 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f8f54640 1 -- 172.21.15.187:0/2690677328 wait complete. 2023-09-12T14:54:34.184 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f8f54640 1 Processor -- start 2023-09-12T14:54:34.184 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f8f54640 1 -- start start 2023-09-12T14:54:34.184 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f8f54640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04f406fdd0 0x7f04f4161200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:34.185 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f8f54640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04f40b2100 0x7f04f4150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:34.185 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f8f54640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04f41556e0 0x7f04f40cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:34.185 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f8f54640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f4151490 con 0x7f04f41556e0 2023-09-12T14:54:34.185 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f8f54640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f40cb9a0 con 0x7f04f406fdd0 2023-09-12T14:54:34.185 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f8f54640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f4160d70 con 0x7f04f40b2100 2023-09-12T14:54:34.186 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f1d9b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04f40b2100 0x7f04f4150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:34.186 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f259c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04f406fdd0 0x7f04f4161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:34.186 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f259c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04f406fdd0 0x7f04f4161200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:48692/0 (socket says 172.21.15.187:48692) 2023-09-12T14:54:34.186 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f1d9b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04f40b2100 0x7f04f4150aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:39794/0 (socket says 172.21.15.187:39794) 2023-09-12T14:54:34.187 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f259c640 1 -- 172.21.15.187:0/3737872766 learned_addr learned my addr 172.21.15.187:0/3737872766 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-09-12T14:54:34.187 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f2d9d640 1 --2- 172.21.15.187:0/3737872766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04f41556e0 0x7f04f40cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:34.187 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f259c640 1 -- 172.21.15.187:0/3737872766 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04f40b2100 msgr2=0x7f04f4150aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:34.187 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f259c640 1 --2- 172.21.15.187:0/3737872766 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04f40b2100 0x7f04f4150aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.187 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f259c640 1 -- 172.21.15.187:0/3737872766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04f41556e0 msgr2=0x7f04f40cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:34.188 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f259c640 1 --2- 172.21.15.187:0/3737872766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04f41556e0 0x7f04f40cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.188 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f259c640 1 -- 172.21.15.187:0/3737872766 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e80094a0 con 0x7f04f406fdd0 2023-09-12T14:54:34.188 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f1d9b640 1 --2- 172.21.15.187:0/3737872766 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04f40b2100 0x7f04f4150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T14:54:34.188 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f2d9d640 1 --2- 172.21.15.187:0/3737872766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04f41556e0 0x7f04f40cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:54:34.188 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f259c640 1 --2- 172.21.15.187:0/3737872766 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04f406fdd0 0x7f04f4161200 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f04e800e970 tx=0x7f04e800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:34.188 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04e2ffd640 1 -- 172.21.15.187:0/3737872766 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e800cd80 con 0x7f04f406fdd0 2023-09-12T14:54:34.189 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04e2ffd640 1 -- 172.21.15.187:0/3737872766 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f04e8004020 con 0x7f04f406fdd0 2023-09-12T14:54:34.189 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f8f54640 1 -- 172.21.15.187:0/3737872766 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04ec009820 con 0x7f04f406fdd0 2023-09-12T14:54:34.189 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04e2ffd640 1 -- 172.21.15.187:0/3737872766 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e800cee0 con 0x7f04f406fdd0 2023-09-12T14:54:34.189 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04f8f54640 1 -- 172.21.15.187:0/3737872766 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f4160760 con 0x7f04f406fdd0 2023-09-12T14:54:34.189 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04e2ffd640 1 -- 172.21.15.187:0/3737872766 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f04e80047d0 con 0x7f04f406fdd0 2023-09-12T14:54:34.190 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.178+0000 7f04e2ffd640 1 -- 172.21.15.187:0/3737872766 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f04e8014070 con 0x7f04f406fdd0 2023-09-12T14:54:34.190 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.182+0000 7f04f8f54640 1 -- 172.21.15.187:0/3737872766 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04f41605c0 con 0x7f04f406fdd0 2023-09-12T14:54:34.190 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.186+0000 7f04e2ffd640 1 -- 172.21.15.187:0/3737872766 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f04f41605c0 con 0x7f04f406fdd0 2023-09-12T14:54:34.254 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.250+0000 7f04f8f54640 1 -- 172.21.15.187:0/3737872766 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "14ba94e7-71fd-48c3-a92e-04b6274992fe", "id": 2} v 0) v1 -- 0x7f04f4155b10 con 0x7f04f406fdd0 2023-09-12T14:54:34.262 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.258+0000 7f04e2ffd640 1 -- 172.21.15.187:0/3737872766 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "14ba94e7-71fd-48c3-a92e-04b6274992fe", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f04e80053e0 con 0x7f04f406fdd0 2023-09-12T14:54:34.262 INFO:teuthology.orchestra.run.smithi187.stdout:2 2023-09-12T14:54:34.263 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.258+0000 7f04f8f54640 1 -- 172.21.15.187:0/3737872766 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04f406fdd0 msgr2=0x7f04f4161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:34.264 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.258+0000 7f04f8f54640 1 --2- 172.21.15.187:0/3737872766 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04f406fdd0 0x7f04f4161200 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f04e800e970 tx=0x7f04e800ee30 comp rx=0 tx=0).stop 2023-09-12T14:54:34.264 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.258+0000 7f04f8f54640 1 -- 172.21.15.187:0/3737872766 shutdown_connections 2023-09-12T14:54:34.264 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.258+0000 7f04f8f54640 1 --2- 172.21.15.187:0/3737872766 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04f40b2100 0x7f04f4150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.264 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.258+0000 7f04f8f54640 1 --2- 172.21.15.187:0/3737872766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04f41556e0 0x7f04f40cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.264 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.258+0000 7f04f8f54640 1 --2- 172.21.15.187:0/3737872766 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04f406fdd0 0x7f04f4161200 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.265 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.258+0000 7f04f8f54640 1 -- 172.21.15.187:0/3737872766 >> 172.21.15.187:0/3737872766 conn(0x7f04f406a950 msgr2=0x7f04f4152c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:34.265 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.262+0000 7f04f8f54640 1 -- 172.21.15.187:0/3737872766 shutdown_connections 2023-09-12T14:54:34.265 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.262+0000 7f04f8f54640 1 -- 172.21.15.187:0/3737872766 wait complete. 2023-09-12T14:54:34.278 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 85179676-991f-4155-9378-73d9e6fed59e 3 2023-09-12T14:54:34.395 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.390+0000 7fd981178640 1 -- 172.21.15.187:0/4085315838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd97c06a870 msgr2=0x7fd97c0c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:34.395 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.390+0000 7fd981178640 1 --2- 172.21.15.187:0/4085315838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd97c06a870 0x7fd97c0c9b30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd97c06ee40 tx=0x7fd96c01c810 comp rx=0 tx=0).stop 2023-09-12T14:54:34.395 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.390+0000 7fd981178640 1 -- 172.21.15.187:0/4085315838 shutdown_connections 2023-09-12T14:54:34.396 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.390+0000 7fd981178640 1 --2- 172.21.15.187:0/4085315838 >> v2:172.21.15.131:3300/0 conn(0x7fd97c070e20 0x7fd97c14fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.396 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.390+0000 7fd981178640 1 --2- 172.21.15.187:0/4085315838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd97c06a870 0x7fd97c0c9b30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.396 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.390+0000 7fd981178640 1 -- 172.21.15.187:0/4085315838 >> 172.21.15.187:0/4085315838 conn(0x7fd97c06b3a0 msgr2=0x7fd97c06b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:34.396 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.390+0000 7fd981178640 1 -- 172.21.15.187:0/4085315838 shutdown_connections 2023-09-12T14:54:34.396 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.390+0000 7fd981178640 1 -- 172.21.15.187:0/4085315838 wait complete. 2023-09-12T14:54:34.397 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd981178640 1 Processor -- start 2023-09-12T14:54:34.397 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd981178640 1 -- start start 2023-09-12T14:54:34.400 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd981178640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd97c06a870 0x7fd97c1f3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:34.400 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd981178640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd97c0709b0 0x7fd97c14a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:34.400 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd981178640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd97c070e20 0x7fd97c14ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:34.400 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd981178640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c0ae050 con 0x7fd97c06a870 2023-09-12T14:54:34.400 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd981178640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c1f3b50 con 0x7fd97c0709b0 2023-09-12T14:54:34.401 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd981178640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c1482d0 con 0x7fd97c070e20 2023-09-12T14:54:34.401 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd97a59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd97c0709b0 0x7fd97c14a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:34.401 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd97a59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd97c0709b0 0x7fd97c14a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:48710/0 (socket says 172.21.15.187:48710) 2023-09-12T14:54:34.401 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd97b59e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd97c070e20 0x7fd97c14ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:34.401 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd97a59c640 1 -- 172.21.15.187:0/792720708 learned_addr learned my addr 172.21.15.187:0/792720708 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-09-12T14:54:34.402 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd97ad9d640 1 --2- 172.21.15.187:0/792720708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd97c06a870 0x7fd97c1f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:34.402 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd97a59c640 1 -- 172.21.15.187:0/792720708 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd97c070e20 msgr2=0x7fd97c14ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:34.402 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd97a59c640 1 --2- 172.21.15.187:0/792720708 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd97c070e20 0x7fd97c14ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.402 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd97a59c640 1 -- 172.21.15.187:0/792720708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd97c06a870 msgr2=0x7fd97c1f3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:34.402 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd97a59c640 1 --2- 172.21.15.187:0/792720708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd97c06a870 0x7fd97c1f3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.403 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd97a59c640 1 -- 172.21.15.187:0/792720708 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd96c009560 con 0x7fd97c0709b0 2023-09-12T14:54:34.403 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd97b59e640 1 --2- 172.21.15.187:0/792720708 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd97c070e20 0x7fd97c14ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T14:54:34.403 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd97ad9d640 1 --2- 172.21.15.187:0/792720708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd97c06a870 0x7fd97c1f3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:54:34.403 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd97a59c640 1 --2- 172.21.15.187:0/792720708 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd97c0709b0 0x7fd97c14a740 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd97000c500 tx=0x7fd97000c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:34.403 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd9637fe640 1 -- 172.21.15.187:0/792720708 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd970005220 con 0x7fd97c0709b0 2023-09-12T14:54:34.404 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd9637fe640 1 -- 172.21.15.187:0/792720708 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd970005470 con 0x7fd97c0709b0 2023-09-12T14:54:34.404 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd981178640 1 -- 172.21.15.187:0/792720708 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97c15a1b0 con 0x7fd97c0709b0 2023-09-12T14:54:34.404 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd9637fe640 1 -- 172.21.15.187:0/792720708 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd970007290 con 0x7fd97c0709b0 2023-09-12T14:54:34.404 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd981178640 1 -- 172.21.15.187:0/792720708 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd97c0add70 con 0x7fd97c0709b0 2023-09-12T14:54:34.404 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd9637fe640 1 -- 172.21.15.187:0/792720708 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd9700067e0 con 0x7fd97c0709b0 2023-09-12T14:54:34.405 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.394+0000 7fd9637fe640 1 -- 172.21.15.187:0/792720708 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fd970012630 con 0x7fd97c0709b0 2023-09-12T14:54:34.405 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.398+0000 7fd981178640 1 -- 172.21.15.187:0/792720708 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd97c073580 con 0x7fd97c0709b0 2023-09-12T14:54:34.405 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.398+0000 7fd9637fe640 1 -- 172.21.15.187:0/792720708 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fd970016020 con 0x7fd97c0709b0 2023-09-12T14:54:34.465 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.462+0000 7fd981178640 1 -- 172.21.15.187:0/792720708 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "85179676-991f-4155-9378-73d9e6fed59e", "id": 3} v 0) v1 -- 0x7fd97c06c410 con 0x7fd97c0709b0 2023-09-12T14:54:34.476 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.474+0000 7fd9637fe640 1 -- 172.21.15.187:0/792720708 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "85179676-991f-4155-9378-73d9e6fed59e", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd97002a0b0 con 0x7fd97c0709b0 2023-09-12T14:54:34.477 INFO:teuthology.orchestra.run.smithi187.stdout:3 2023-09-12T14:54:34.478 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.474+0000 7fd981178640 1 -- 172.21.15.187:0/792720708 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd97c0709b0 msgr2=0x7fd97c14a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:34.478 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.474+0000 7fd981178640 1 --2- 172.21.15.187:0/792720708 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd97c0709b0 0x7fd97c14a740 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd97000c500 tx=0x7fd97000c9c0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.478 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.474+0000 7fd981178640 1 -- 172.21.15.187:0/792720708 shutdown_connections 2023-09-12T14:54:34.479 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.474+0000 7fd981178640 1 --2- 172.21.15.187:0/792720708 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd97c070e20 0x7fd97c14ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.479 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.474+0000 7fd981178640 1 --2- 172.21.15.187:0/792720708 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd97c0709b0 0x7fd97c14a740 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.479 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.474+0000 7fd981178640 1 --2- 172.21.15.187:0/792720708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd97c06a870 0x7fd97c1f3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.479 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.474+0000 7fd981178640 1 -- 172.21.15.187:0/792720708 >> 172.21.15.187:0/792720708 conn(0x7fd97c06b3a0 msgr2=0x7fd97c06d170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:34.479 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.474+0000 7fd981178640 1 -- 172.21.15.187:0/792720708 shutdown_connections 2023-09-12T14:54:34.479 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.474+0000 7fd981178640 1 -- 172.21.15.187:0/792720708 wait complete. 2023-09-12T14:54:34.493 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 1b65089f-9b5f-4133-8997-e06bf12287a8 4 2023-09-12T14:54:34.574 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-09-12T14:54:34.570+0000 7f19e66bb180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-12T14:54:34.614 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.610+0000 7fa1ce152640 1 -- 172.21.15.187:0/1082933610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1c806fdd0 msgr2=0x7fa1c8070240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:34.614 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.610+0000 7fa1ce152640 1 --2- 172.21.15.187:0/1082933610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1c806fdd0 0x7fa1c8070240 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa1c806e980 tx=0x7fa1c001ccf0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.614 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.610+0000 7fa1ce152640 1 -- 172.21.15.187:0/1082933610 shutdown_connections 2023-09-12T14:54:34.614 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.610+0000 7fa1ce152640 1 --2- 172.21.15.187:0/1082933610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1c806fdd0 0x7fa1c8070240 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.615 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.610+0000 7fa1ce152640 1 --2- 172.21.15.187:0/1082933610 >> v2:172.21.15.131:3300/0 conn(0x7fa1c81556e0 0x7fa1c8158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.615 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.610+0000 7fa1ce152640 1 -- 172.21.15.187:0/1082933610 >> 172.21.15.187:0/1082933610 conn(0x7fa1c806a950 msgr2=0x7fa1c80af5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:34.615 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.610+0000 7fa1ce152640 1 -- 172.21.15.187:0/1082933610 shutdown_connections 2023-09-12T14:54:34.615 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.610+0000 7fa1ce152640 1 -- 172.21.15.187:0/1082933610 wait complete. 2023-09-12T14:54:34.616 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.610+0000 7fa1ce152640 1 Processor -- start 2023-09-12T14:54:34.616 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.610+0000 7fa1ce152640 1 -- start start 2023-09-12T14:54:34.617 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.614+0000 7fa1ce152640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1c806fdd0 0x7fa1c8152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:34.617 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.614+0000 7fa1ce152640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1c80b2100 0x7fa1c81515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:34.617 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.614+0000 7fa1ce152640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1c81556e0 0x7fa1c8160b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:34.617 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.614+0000 7fa1ce152640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c8151b30 con 0x7fa1c81556e0 2023-09-12T14:54:34.618 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.614+0000 7fa1ce152640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c8161040 con 0x7fa1c806fdd0 2023-09-12T14:54:34.618 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.614+0000 7fa1ce152640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c81612c0 con 0x7fa1c80b2100 2023-09-12T14:54:34.618 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-09-12T14:54:34.614+0000 7f0f107e2180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-12T14:54:34.620 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.614+0000 7fa1c77fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1c806fdd0 0x7fa1c8152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:34.621 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.614+0000 7fa1c77fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1c806fdd0 0x7fa1c8152190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:48740/0 (socket says 172.21.15.187:48740) 2023-09-12T14:54:34.621 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.614+0000 7fa1c77fe640 1 -- 172.21.15.187:0/3744840478 learned_addr learned my addr 172.21.15.187:0/3744840478 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-09-12T14:54:34.621 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.614+0000 7fa1c6ffd640 1 --2- 172.21.15.187:0/3744840478 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1c80b2100 0x7fa1c81515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:34.621 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.614+0000 7fa1c7fff640 1 --2- 172.21.15.187:0/3744840478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1c81556e0 0x7fa1c8160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:34.622 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.614+0000 7fa1c77fe640 1 -- 172.21.15.187:0/3744840478 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1c80b2100 msgr2=0x7fa1c81515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:34.622 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.614+0000 7fa1c77fe640 1 --2- 172.21.15.187:0/3744840478 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1c80b2100 0x7fa1c81515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.622 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.614+0000 7fa1c77fe640 1 -- 172.21.15.187:0/3744840478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1c81556e0 msgr2=0x7fa1c8160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:34.622 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.614+0000 7fa1c77fe640 1 --2- 172.21.15.187:0/3744840478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1c81556e0 0x7fa1c8160b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.623 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.614+0000 7fa1c77fe640 1 -- 172.21.15.187:0/3744840478 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c00098e0 con 0x7fa1c806fdd0 2023-09-12T14:54:34.623 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.614+0000 7fa1c77fe640 1 --2- 172.21.15.187:0/3744840478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1c806fdd0 0x7fa1c8152190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa1b800b440 tx=0x7fa1b800b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:34.628 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.614+0000 7fa1affff640 1 -- 172.21.15.187:0/3744840478 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b8011890 con 0x7fa1c806fdd0 2023-09-12T14:54:34.628 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.614+0000 7fa1affff640 1 -- 172.21.15.187:0/3744840478 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa1b8011b00 con 0x7fa1c806fdd0 2023-09-12T14:54:34.628 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.614+0000 7fa1ce152640 1 -- 172.21.15.187:0/3744840478 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1c81615a0 con 0x7fa1c806fdd0 2023-09-12T14:54:34.628 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.614+0000 7fa1affff640 1 -- 172.21.15.187:0/3744840478 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b80103e0 con 0x7fa1c806fdd0 2023-09-12T14:54:34.629 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.614+0000 7fa1ce152640 1 -- 172.21.15.187:0/3744840478 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c815ccb0 con 0x7fa1c806fdd0 2023-09-12T14:54:34.629 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.614+0000 7fa1affff640 1 -- 172.21.15.187:0/3744840478 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa1b8010840 con 0x7fa1c806fdd0 2023-09-12T14:54:34.629 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.614+0000 7fa1affff640 1 -- 172.21.15.187:0/3744840478 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fa1b801a400 con 0x7fa1c806fdd0 2023-09-12T14:54:34.629 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.614+0000 7fa1ce152640 1 -- 172.21.15.187:0/3744840478 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1c815ced0 con 0x7fa1c806fdd0 2023-09-12T14:54:34.629 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.614+0000 7fa1affff640 1 -- 172.21.15.187:0/3744840478 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fa1b8015020 con 0x7fa1c806fdd0 2023-09-12T14:54:34.630 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-09-12T14:54:34.622+0000 7f19e66bb180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-12T14:54:34.671 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-09-12T14:54:34.666+0000 7f0f107e2180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-12T14:54:34.680 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-09-12T14:54:34.674+0000 7f19e66bb180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-12T14:54:34.687 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.682+0000 7fa1ce152640 1 -- 172.21.15.187:0/3744840478 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1b65089f-9b5f-4133-8997-e06bf12287a8", "id": 4} v 0) v1 -- 0x7fa1c8155b40 con 0x7fa1c806fdd0 2023-09-12T14:54:34.694 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.690+0000 7fa1affff640 1 -- 172.21.15.187:0/3744840478 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1b65089f-9b5f-4133-8997-e06bf12287a8", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa1b8010bc0 con 0x7fa1c806fdd0 2023-09-12T14:54:34.694 INFO:teuthology.orchestra.run.smithi187.stdout:4 2023-09-12T14:54:34.696 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.694+0000 7fa1ce152640 1 -- 172.21.15.187:0/3744840478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1c806fdd0 msgr2=0x7fa1c8152190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:34.696 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.694+0000 7fa1ce152640 1 --2- 172.21.15.187:0/3744840478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1c806fdd0 0x7fa1c8152190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa1b800b440 tx=0x7fa1b800b900 comp rx=0 tx=0).stop 2023-09-12T14:54:34.696 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.694+0000 7fa1ce152640 1 -- 172.21.15.187:0/3744840478 shutdown_connections 2023-09-12T14:54:34.696 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.694+0000 7fa1ce152640 1 --2- 172.21.15.187:0/3744840478 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1c80b2100 0x7fa1c81515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.696 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.694+0000 7fa1ce152640 1 --2- 172.21.15.187:0/3744840478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1c81556e0 0x7fa1c8160b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.697 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.694+0000 7fa1ce152640 1 --2- 172.21.15.187:0/3744840478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1c806fdd0 0x7fa1c8152190 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.697 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.694+0000 7fa1ce152640 1 -- 172.21.15.187:0/3744840478 >> 172.21.15.187:0/3744840478 conn(0x7fa1c806a950 msgr2=0x7fa1c806be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:34.697 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.694+0000 7fa1ce152640 1 -- 172.21.15.187:0/3744840478 shutdown_connections 2023-09-12T14:54:34.697 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.694+0000 7fa1ce152640 1 -- 172.21.15.187:0/3744840478 wait complete. 2023-09-12T14:54:34.711 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new cd4abd3e-97a3-4bd6-82b9-c531b6cf266c 5 2023-09-12T14:54:34.726 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-09-12T14:54:34.722+0000 7f0f107e2180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-12T14:54:34.757 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-09-12T14:54:34.754+0000 7f19e66bb180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-12T14:54:34.805 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-09-12T14:54:34.802+0000 7f0f107e2180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-12T14:54:34.808 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-09-12T14:54:34.802+0000 7f19e66bb180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-12T14:54:34.828 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f24449b0640 1 -- 172.21.15.187:0/1067035041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24401556e0 msgr2=0x7f2440158130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:34.828 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f24449b0640 1 --2- 172.21.15.187:0/1067035041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24401556e0 0x7f2440158130 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2430009840 tx=0x7f243001c7d0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.828 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f24449b0640 1 -- 172.21.15.187:0/1067035041 shutdown_connections 2023-09-12T14:54:34.829 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f24449b0640 1 --2- 172.21.15.187:0/1067035041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f244006fdd0 0x7f2440070240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.829 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f24449b0640 1 --2- 172.21.15.187:0/1067035041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24401556e0 0x7f2440158130 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.829 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f24449b0640 1 -- 172.21.15.187:0/1067035041 >> 172.21.15.187:0/1067035041 conn(0x7f244006a950 msgr2=0x7f24400af5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:34.829 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f24449b0640 1 -- 172.21.15.187:0/1067035041 shutdown_connections 2023-09-12T14:54:34.829 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f24449b0640 1 -- 172.21.15.187:0/1067035041 wait complete. 2023-09-12T14:54:34.831 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f24449b0640 1 Processor -- start 2023-09-12T14:54:34.832 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f24449b0640 1 -- start start 2023-09-12T14:54:34.832 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f24449b0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f244006fdd0 0x7f2440154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:34.832 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f24449b0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24400b2100 0x7f2440152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:34.832 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f24449b0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24401556e0 0x7f244015c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:34.833 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f24449b0640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2440153a00 con 0x7f24401556e0 2023-09-12T14:54:34.833 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f24449b0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24401f4460 con 0x7f24400b2100 2023-09-12T14:54:34.833 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f24449b0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24400cbd90 con 0x7f244006fdd0 2023-09-12T14:54:34.833 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f243e59c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f244006fdd0 0x7f2440154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:34.833 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f2437fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24400b2100 0x7f2440152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:34.834 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f243e59c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f244006fdd0 0x7f2440154420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:39824/0 (socket says 172.21.15.187:39824) 2023-09-12T14:54:34.834 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f243e59c640 1 -- 172.21.15.187:0/1463251384 learned_addr learned my addr 172.21.15.187:0/1463251384 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-09-12T14:54:34.834 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f243ed9d640 1 --2- 172.21.15.187:0/1463251384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24401556e0 0x7f244015c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:34.834 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f2437fff640 1 -- 172.21.15.187:0/1463251384 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f244006fdd0 msgr2=0x7f2440154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:34.834 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f2437fff640 1 --2- 172.21.15.187:0/1463251384 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f244006fdd0 0x7f2440154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.835 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f2437fff640 1 -- 172.21.15.187:0/1463251384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24401556e0 msgr2=0x7f244015c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:34.835 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f2437fff640 1 --2- 172.21.15.187:0/1463251384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24401556e0 0x7f244015c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.835 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f2437fff640 1 -- 172.21.15.187:0/1463251384 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24380098e0 con 0x7f24400b2100 2023-09-12T14:54:34.835 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f243e59c640 1 --2- 172.21.15.187:0/1463251384 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f244006fdd0 0x7f2440154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T14:54:34.835 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f2437fff640 1 --2- 172.21.15.187:0/1463251384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24400b2100 0x7f2440152950 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f242800c3c0 tx=0x7f242800c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:34.836 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f2436ffd640 1 -- 172.21.15.187:0/1463251384 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24280052d0 con 0x7f24400b2100 2023-09-12T14:54:34.836 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f2436ffd640 1 -- 172.21.15.187:0/1463251384 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2428007020 con 0x7f24400b2100 2023-09-12T14:54:34.836 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f24449b0640 1 -- 172.21.15.187:0/1463251384 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24300094a0 con 0x7f24400b2100 2023-09-12T14:54:34.836 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f2436ffd640 1 -- 172.21.15.187:0/1463251384 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2428007470 con 0x7f24400b2100 2023-09-12T14:54:34.836 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f24449b0640 1 -- 172.21.15.187:0/1463251384 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24401f46b0 con 0x7f24400b2100 2023-09-12T14:54:34.837 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f2436ffd640 1 -- 172.21.15.187:0/1463251384 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2428006ae0 con 0x7f24400b2100 2023-09-12T14:54:34.837 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.826+0000 7f2436ffd640 1 -- 172.21.15.187:0/1463251384 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f2428003ab0 con 0x7f24400b2100 2023-09-12T14:54:34.837 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.830+0000 7f24449b0640 1 -- 172.21.15.187:0/1463251384 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2440073580 con 0x7f24400b2100 2023-09-12T14:54:34.837 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.830+0000 7f2436ffd640 1 -- 172.21.15.187:0/1463251384 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f2428020060 con 0x7f24400b2100 2023-09-12T14:54:34.862 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-09-12T14:54:34.858+0000 7f0f107e2180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-12T14:54:34.884 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-09-12T14:54:34.878+0000 7f19e66bb180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-12T14:54:34.903 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.898+0000 7f24449b0640 1 -- 172.21.15.187:0/1463251384 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cd4abd3e-97a3-4bd6-82b9-c531b6cf266c", "id": 5} v 0) v1 -- 0x7f244006e470 con 0x7f24400b2100 2023-09-12T14:54:34.911 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.906+0000 7f2436ffd640 1 -- 172.21.15.187:0/1463251384 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cd4abd3e-97a3-4bd6-82b9-c531b6cf266c", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2428006d90 con 0x7f24400b2100 2023-09-12T14:54:34.911 INFO:teuthology.orchestra.run.smithi187.stdout:5 2023-09-12T14:54:34.912 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.910+0000 7f24449b0640 1 -- 172.21.15.187:0/1463251384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24400b2100 msgr2=0x7f2440152950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:34.912 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.910+0000 7f24449b0640 1 --2- 172.21.15.187:0/1463251384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24400b2100 0x7f2440152950 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f242800c3c0 tx=0x7f242800c880 comp rx=0 tx=0).stop 2023-09-12T14:54:34.912 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.910+0000 7f24449b0640 1 -- 172.21.15.187:0/1463251384 shutdown_connections 2023-09-12T14:54:34.913 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.910+0000 7f24449b0640 1 --2- 172.21.15.187:0/1463251384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24401556e0 0x7f244015c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.913 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.910+0000 7f24449b0640 1 --2- 172.21.15.187:0/1463251384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24400b2100 0x7f2440152950 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.913 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.910+0000 7f24449b0640 1 --2- 172.21.15.187:0/1463251384 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f244006fdd0 0x7f2440154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:34.913 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.910+0000 7f24449b0640 1 -- 172.21.15.187:0/1463251384 >> 172.21.15.187:0/1463251384 conn(0x7f244006a950 msgr2=0x7f244006be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:34.913 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.910+0000 7f24449b0640 1 -- 172.21.15.187:0/1463251384 shutdown_connections 2023-09-12T14:54:34.914 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:34.910+0000 7f24449b0640 1 -- 172.21.15.187:0/1463251384 wait complete. 2023-09-12T14:54:34.927 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 53eed235-0d36-4830-a8c0-c2e19fe46d46 6 2023-09-12T14:54:34.935 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-09-12T14:54:34.930+0000 7f19e66bb180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-12T14:54:34.942 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-09-12T14:54:34.938+0000 7f0f107e2180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-12T14:54:34.996 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-09-12T14:54:34.994+0000 7f0f107e2180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-12T14:54:35.060 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f28402bf640 1 -- 172.21.15.187:0/2410893388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f283815bdc0 msgr2=0x7f2838070790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:35.060 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f28402bf640 1 --2- 172.21.15.187:0/2410893388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f283815bdc0 0x7f2838070790 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f28380c9a90 tx=0x7f283001cc90 comp rx=0 tx=0).stop 2023-09-12T14:54:35.060 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f28402bf640 1 -- 172.21.15.187:0/2410893388 shutdown_connections 2023-09-12T14:54:35.061 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f28402bf640 1 --2- 172.21.15.187:0/2410893388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f283815bdc0 0x7f2838070790 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:35.061 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f28402bf640 1 --2- 172.21.15.187:0/2410893388 >> v2:172.21.15.131:3300/0 conn(0x7f28381578a0 0x7f28380ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:35.061 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f28402bf640 1 -- 172.21.15.187:0/2410893388 >> 172.21.15.187:0/2410893388 conn(0x7f283806b280 msgr2=0x7f283806b690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:35.061 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f28402bf640 1 -- 172.21.15.187:0/2410893388 shutdown_connections 2023-09-12T14:54:35.062 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f28402bf640 1 -- 172.21.15.187:0/2410893388 wait complete. 2023-09-12T14:54:35.064 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f28402bf640 1 Processor -- start 2023-09-12T14:54:35.064 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f28402bf640 1 -- start start 2023-09-12T14:54:35.064 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f28402bf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28380c9670 0x7f283814de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:35.065 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f28402bf640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28381578a0 0x7f283814e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:35.065 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f28402bf640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f283815bdc0 0x7f283814eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:35.065 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f28402bf640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283814e340 con 0x7f28380c9670 2023-09-12T14:54:35.065 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f28402bf640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28381561e0 con 0x7f28381578a0 2023-09-12T14:54:35.065 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f28402bf640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28380cbd80 con 0x7f283815bdc0 2023-09-12T14:54:35.066 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f283d85a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28381578a0 0x7f283814e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:35.066 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f283e85c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f283815bdc0 0x7f283814eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:35.066 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f283d85a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28381578a0 0x7f283814e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:48772/0 (socket says 172.21.15.187:48772) 2023-09-12T14:54:35.066 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f283d85a640 1 -- 172.21.15.187:0/1768284381 learned_addr learned my addr 172.21.15.187:0/1768284381 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-09-12T14:54:35.067 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f283e85c640 1 --2- 172.21.15.187:0/1768284381 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f283815bdc0 0x7f283814eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:39832/0 (socket says 172.21.15.187:39832) 2023-09-12T14:54:35.067 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f283e05b640 1 --2- 172.21.15.187:0/1768284381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28380c9670 0x7f283814de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:35.067 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f283d85a640 1 -- 172.21.15.187:0/1768284381 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f283815bdc0 msgr2=0x7f283814eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:35.067 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f283d85a640 1 --2- 172.21.15.187:0/1768284381 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f283815bdc0 0x7f283814eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:35.067 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f283d85a640 1 -- 172.21.15.187:0/1768284381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28380c9670 msgr2=0x7f283814de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:35.068 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f283d85a640 1 --2- 172.21.15.187:0/1768284381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28380c9670 0x7f283814de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:35.068 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f283d85a640 1 -- 172.21.15.187:0/1768284381 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28300098e0 con 0x7f28381578a0 2023-09-12T14:54:35.068 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f283e85c640 1 --2- 172.21.15.187:0/1768284381 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f283815bdc0 0x7f283814eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T14:54:35.068 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f283e05b640 1 --2- 172.21.15.187:0/1768284381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28380c9670 0x7f283814de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:54:35.068 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f283d85a640 1 --2- 172.21.15.187:0/1768284381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28381578a0 0x7f283814e5a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f282800c450 tx=0x7f282800c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:35.069 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f282effd640 1 -- 172.21.15.187:0/1768284381 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2828012830 con 0x7f28381578a0 2023-09-12T14:54:35.069 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f282effd640 1 -- 172.21.15.187:0/1768284381 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2828012aa0 con 0x7f28381578a0 2023-09-12T14:54:35.069 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f28402bf640 1 -- 172.21.15.187:0/1768284381 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28381555d0 con 0x7f28381578a0 2023-09-12T14:54:35.069 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f282effd640 1 -- 172.21.15.187:0/1768284381 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2828011380 con 0x7f28381578a0 2023-09-12T14:54:35.069 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f28402bf640 1 -- 172.21.15.187:0/1768284381 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2838155b90 con 0x7f28381578a0 2023-09-12T14:54:35.070 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f282effd640 1 -- 172.21.15.187:0/1768284381 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f28280117e0 con 0x7f28381578a0 2023-09-12T14:54:35.070 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.058+0000 7f282effd640 1 -- 172.21.15.187:0/1768284381 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f282801b3c0 con 0x7f28381578a0 2023-09-12T14:54:35.070 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.062+0000 7f282cff9640 1 -- 172.21.15.187:0/1768284381 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2814005260 con 0x7f28381578a0 2023-09-12T14:54:35.071 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.062+0000 7f282effd640 1 -- 172.21.15.187:0/1768284381 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f282801c460 con 0x7f28381578a0 2023-09-12T14:54:35.073 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-09-12T14:54:35.070+0000 7f19e66bb180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-12T14:54:35.132 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-09-12T14:54:35.126+0000 7f19e66bb180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-12T14:54:35.133 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.130+0000 7f282cff9640 1 -- 172.21.15.187:0/1768284381 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "53eed235-0d36-4830-a8c0-c2e19fe46d46", "id": 6} v 0) v1 -- 0x7f28140050d0 con 0x7f28381578a0 2023-09-12T14:54:35.141 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-09-12T14:54:35.138+0000 7f0f107e2180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-12T14:54:35.143 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.142+0000 7f282effd640 1 -- 172.21.15.187:0/1768284381 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "53eed235-0d36-4830-a8c0-c2e19fe46d46", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2828011c70 con 0x7f28381578a0 2023-09-12T14:54:35.143 INFO:teuthology.orchestra.run.smithi187.stdout:6 2023-09-12T14:54:35.145 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.142+0000 7f282cff9640 1 -- 172.21.15.187:0/1768284381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28381578a0 msgr2=0x7f283814e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:35.145 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.142+0000 7f282cff9640 1 --2- 172.21.15.187:0/1768284381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28381578a0 0x7f283814e5a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f282800c450 tx=0x7f282800c910 comp rx=0 tx=0).stop 2023-09-12T14:54:35.145 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.142+0000 7f282cff9640 1 -- 172.21.15.187:0/1768284381 shutdown_connections 2023-09-12T14:54:35.145 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.142+0000 7f282cff9640 1 --2- 172.21.15.187:0/1768284381 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f283815bdc0 0x7f283814eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:35.145 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.142+0000 7f282cff9640 1 --2- 172.21.15.187:0/1768284381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28381578a0 0x7f283814e5a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:35.146 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.142+0000 7f282cff9640 1 --2- 172.21.15.187:0/1768284381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28380c9670 0x7f283814de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:35.146 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.142+0000 7f282cff9640 1 -- 172.21.15.187:0/1768284381 >> 172.21.15.187:0/1768284381 conn(0x7f283806b280 msgr2=0x7f28380acc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:35.146 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.142+0000 7f282cff9640 1 -- 172.21.15.187:0/1768284381 shutdown_connections 2023-09-12T14:54:35.146 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.142+0000 7f282cff9640 1 -- 172.21.15.187:0/1768284381 wait complete. 2023-09-12T14:54:35.162 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 9d511448-aac0-4c4b-99ed-221457021871 7 2023-09-12T14:54:35.206 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-09-12T14:54:35.202+0000 7f0f107e2180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-12T14:54:35.283 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.278+0000 7fb33cede640 1 -- 172.21.15.187:0/3362709371 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3240039d0 msgr2=0x7fb324003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:35.283 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.278+0000 7fb33cede640 1 --2- 172.21.15.187:0/3362709371 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3240039d0 0x7fb324003df0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb3200016b0 tx=0x7fb32001c780 comp rx=0 tx=0).stop 2023-09-12T14:54:35.283 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.278+0000 7fb33cede640 1 -- 172.21.15.187:0/3362709371 shutdown_connections 2023-09-12T14:54:35.283 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.278+0000 7fb33cede640 1 --2- 172.21.15.187:0/3362709371 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3240039d0 0x7fb324003df0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:35.283 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.278+0000 7fb33cede640 1 -- 172.21.15.187:0/3362709371 >> 172.21.15.187:0/3362709371 conn(0x7fb3380cdd80 msgr2=0x7fb3380ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:35.284 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.278+0000 7fb33cede640 1 -- 172.21.15.187:0/3362709371 shutdown_connections 2023-09-12T14:54:35.284 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.278+0000 7fb33cede640 1 -- 172.21.15.187:0/3362709371 wait complete. 2023-09-12T14:54:35.286 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.282+0000 7fb33cede640 1 Processor -- start 2023-09-12T14:54:35.286 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.282+0000 7fb33cede640 1 -- start start 2023-09-12T14:54:35.287 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.282+0000 7fb33cede640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3240039d0 0x7fb3381564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:35.287 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.282+0000 7fb33cede640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3380ccc70 0x7fb3380cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:35.287 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.282+0000 7fb33cede640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3380ac010 0x7fb3380ac490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:35.287 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.282+0000 7fb33cede640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3380cd630 con 0x7fb3380ac010 2023-09-12T14:54:35.287 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.282+0000 7fb33cede640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3380ab5e0 con 0x7fb3240039d0 2023-09-12T14:54:35.288 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.282+0000 7fb33cede640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3380aaa10 con 0x7fb3380ccc70 2023-09-12T14:54:35.288 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.282+0000 7fb32ffff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3380ccc70 0x7fb3380cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:35.288 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.282+0000 7fb32ffff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3380ccc70 0x7fb3380cd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:39844/0 (socket says 172.21.15.187:39844) 2023-09-12T14:54:35.288 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.282+0000 7fb32ffff640 1 -- 172.21.15.187:0/882731711 learned_addr learned my addr 172.21.15.187:0/882731711 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-09-12T14:54:35.288 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.282+0000 7fb33659c640 1 --2- 172.21.15.187:0/882731711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3240039d0 0x7fb3381564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:35.288 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.282+0000 7fb336d9d640 1 --2- 172.21.15.187:0/882731711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3380ac010 0x7fb3380ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:35.289 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.282+0000 7fb32ffff640 1 -- 172.21.15.187:0/882731711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3240039d0 msgr2=0x7fb3381564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:35.289 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.282+0000 7fb32ffff640 1 --2- 172.21.15.187:0/882731711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3240039d0 0x7fb3381564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:35.289 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.282+0000 7fb32ffff640 1 -- 172.21.15.187:0/882731711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3380ac010 msgr2=0x7fb3380ac490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:35.289 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.282+0000 7fb32ffff640 1 --2- 172.21.15.187:0/882731711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3380ac010 0x7fb3380ac490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:35.289 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.282+0000 7fb32ffff640 1 -- 172.21.15.187:0/882731711 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb338156a30 con 0x7fb3380ccc70 2023-09-12T14:54:35.290 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.282+0000 7fb32ffff640 1 --2- 172.21.15.187:0/882731711 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3380ccc70 0x7fb3380cd0f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb328006fd0 tx=0x7fb3280066c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:35.290 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.282+0000 7fb32effd640 1 -- 172.21.15.187:0/882731711 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328011840 con 0x7fb3380ccc70 2023-09-12T14:54:35.290 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.282+0000 7fb32effd640 1 -- 172.21.15.187:0/882731711 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb3280119c0 con 0x7fb3380ccc70 2023-09-12T14:54:35.290 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.282+0000 7fb32effd640 1 -- 172.21.15.187:0/882731711 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328011dd0 con 0x7fb3380ccc70 2023-09-12T14:54:35.290 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.282+0000 7fb33cede640 1 -- 172.21.15.187:0/882731711 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3380aacb0 con 0x7fb3380ccc70 2023-09-12T14:54:35.291 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.282+0000 7fb33cede640 1 -- 172.21.15.187:0/882731711 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3380ab270 con 0x7fb3380ccc70 2023-09-12T14:54:35.291 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.282+0000 7fb32effd640 1 -- 172.21.15.187:0/882731711 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb3280103c0 con 0x7fb3380ccc70 2023-09-12T14:54:35.291 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.282+0000 7fb32effd640 1 -- 172.21.15.187:0/882731711 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fb328023a00 con 0x7fb3380ccc70 2023-09-12T14:54:35.291 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.282+0000 7fb33cede640 1 -- 172.21.15.187:0/882731711 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3380ab430 con 0x7fb3380ccc70 2023-09-12T14:54:35.292 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.286+0000 7fb32effd640 1 -- 172.21.15.187:0/882731711 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fb32801f060 con 0x7fb3380ccc70 2023-09-12T14:54:35.357 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.354+0000 7fb33cede640 1 -- 172.21.15.187:0/882731711 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9d511448-aac0-4c4b-99ed-221457021871", "id": 7} v 0) v1 -- 0x7fb338073580 con 0x7fb3380ccc70 2023-09-12T14:54:35.363 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.358+0000 7fb32effd640 1 -- 172.21.15.187:0/882731711 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9d511448-aac0-4c4b-99ed-221457021871", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb328015070 con 0x7fb3380ccc70 2023-09-12T14:54:35.363 INFO:teuthology.orchestra.run.smithi187.stdout:7 2023-09-12T14:54:35.364 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.362+0000 7fb33cede640 1 -- 172.21.15.187:0/882731711 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3380ccc70 msgr2=0x7fb3380cd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:35.364 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.362+0000 7fb33cede640 1 --2- 172.21.15.187:0/882731711 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3380ccc70 0x7fb3380cd0f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb328006fd0 tx=0x7fb3280066c0 comp rx=0 tx=0).stop 2023-09-12T14:54:35.364 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.362+0000 7fb33cede640 1 -- 172.21.15.187:0/882731711 shutdown_connections 2023-09-12T14:54:35.365 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.362+0000 7fb33cede640 1 --2- 172.21.15.187:0/882731711 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3380ccc70 0x7fb3380cd0f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:35.365 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.362+0000 7fb33cede640 1 --2- 172.21.15.187:0/882731711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3380ac010 0x7fb3380ac490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:35.365 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.362+0000 7fb33cede640 1 --2- 172.21.15.187:0/882731711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3240039d0 0x7fb3381564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:35.365 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.362+0000 7fb33cede640 1 -- 172.21.15.187:0/882731711 >> 172.21.15.187:0/882731711 conn(0x7fb3380cdd80 msgr2=0x7fb3380b1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:35.365 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.362+0000 7fb33cede640 1 -- 172.21.15.187:0/882731711 shutdown_connections 2023-09-12T14:54:35.366 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:54:35.362+0000 7fb33cede640 1 -- 172.21.15.187:0/882731711 wait complete. 2023-09-12T14:54:35.379 INFO:tasks.ceph.osd.0:Restarting daemon 2023-09-12T14:54:35.379 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-09-12T14:54:35.382 INFO:tasks.ceph.osd.0:Started 2023-09-12T14:54:35.382 INFO:tasks.ceph.osd.1:Restarting daemon 2023-09-12T14:54:35.382 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-09-12T14:54:35.385 INFO:tasks.ceph.osd.1:Started 2023-09-12T14:54:35.385 INFO:tasks.ceph.osd.2:Restarting daemon 2023-09-12T14:54:35.385 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-09-12T14:54:35.388 INFO:tasks.ceph.osd.2:Started 2023-09-12T14:54:35.388 INFO:tasks.ceph.osd.3:Restarting daemon 2023-09-12T14:54:35.389 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-09-12T14:54:35.391 INFO:tasks.ceph.osd.3:Started 2023-09-12T14:54:35.391 INFO:tasks.ceph.osd.4:Restarting daemon 2023-09-12T14:54:35.391 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-09-12T14:54:35.424 INFO:tasks.ceph.osd.4:Started 2023-09-12T14:54:35.424 INFO:tasks.ceph.osd.5:Restarting daemon 2023-09-12T14:54:35.425 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-09-12T14:54:35.427 INFO:tasks.ceph.osd.5:Started 2023-09-12T14:54:35.427 INFO:tasks.ceph.osd.6:Restarting daemon 2023-09-12T14:54:35.427 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-09-12T14:54:35.429 INFO:tasks.ceph.osd.6:Started 2023-09-12T14:54:35.430 INFO:tasks.ceph.osd.7:Restarting daemon 2023-09-12T14:54:35.430 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-09-12T14:54:35.432 INFO:tasks.ceph.osd.7:Started 2023-09-12T14:54:35.433 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:54:35.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.574+0000 7fef6a68b640 1 -- 172.21.15.131:0/4026438588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef640d1680 msgr2=0x7fef640b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:35.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.574+0000 7fef6a68b640 1 --2- 172.21.15.131:0/4026438588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef640d1680 0x7fef640b0750 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fef54009900 tx=0x7fef5401c890 comp rx=0 tx=0).stop 2023-09-12T14:54:35.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.574+0000 7fef6a68b640 1 -- 172.21.15.131:0/4026438588 shutdown_connections 2023-09-12T14:54:35.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.574+0000 7fef6a68b640 1 --2- 172.21.15.131:0/4026438588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef640d1680 0x7fef640b0750 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:35.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.574+0000 7fef6a68b640 1 -- 172.21.15.131:0/4026438588 >> 172.21.15.131:0/4026438588 conn(0x7fef6409a200 msgr2=0x7fef640d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:35.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.574+0000 7fef6a68b640 1 -- 172.21.15.131:0/4026438588 shutdown_connections 2023-09-12T14:54:35.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.574+0000 7fef6a68b640 1 -- 172.21.15.131:0/4026438588 wait complete. 2023-09-12T14:54:35.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.574+0000 7fef6a68b640 1 Processor -- start 2023-09-12T14:54:35.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.574+0000 7fef6a68b640 1 -- start start 2023-09-12T14:54:35.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.578+0000 7fef6a68b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef640cf2d0 0x7fef640b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:35.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.578+0000 7fef6a68b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef640d1680 0x7fef640cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:35.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.578+0000 7fef6a68b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef6415e9f0 0x7fef640cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:35.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.578+0000 7fef6a68b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef640b43b0 con 0x7fef640cf2d0 2023-09-12T14:54:35.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.578+0000 7fef6a68b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef640c8460 con 0x7fef640d1680 2023-09-12T14:54:35.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.578+0000 7fef6a68b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef640b64f0 con 0x7fef6415e9f0 2023-09-12T14:54:35.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.578+0000 7fef63fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef640cf2d0 0x7fef640b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:35.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.578+0000 7fef63fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef640cf2d0 0x7fef640b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56318/0 (socket says 172.21.15.131:56318) 2023-09-12T14:54:35.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.578+0000 7fef63fff640 1 -- 172.21.15.131:0/3305862254 learned_addr learned my addr 172.21.15.131:0/3305862254 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:35.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.578+0000 7fef637fe640 1 --2- 172.21.15.131:0/3305862254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef640d1680 0x7fef640cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:35.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.578+0000 7fef68c28640 1 --2- 172.21.15.131:0/3305862254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef6415e9f0 0x7fef640cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:35.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.578+0000 7fef63fff640 1 -- 172.21.15.131:0/3305862254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef6415e9f0 msgr2=0x7fef640cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:35.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.578+0000 7fef63fff640 1 --2- 172.21.15.131:0/3305862254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef6415e9f0 0x7fef640cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:35.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.578+0000 7fef63fff640 1 -- 172.21.15.131:0/3305862254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef640d1680 msgr2=0x7fef640cbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:35.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.578+0000 7fef63fff640 1 --2- 172.21.15.131:0/3305862254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef640d1680 0x7fef640cbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:35.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.578+0000 7fef63fff640 1 -- 172.21.15.131:0/3305862254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef54009560 con 0x7fef640cf2d0 2023-09-12T14:54:35.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.578+0000 7fef63fff640 1 --2- 172.21.15.131:0/3305862254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef640cf2d0 0x7fef640b3b60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fef5800c500 tx=0x7fef5800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:35.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.578+0000 7fef60ff9640 1 -- 172.21.15.131:0/3305862254 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef58005220 con 0x7fef640cf2d0 2023-09-12T14:54:35.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.578+0000 7fef60ff9640 1 -- 172.21.15.131:0/3305862254 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fef58005470 con 0x7fef640cf2d0 2023-09-12T14:54:35.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.578+0000 7fef6a68b640 1 -- 172.21.15.131:0/3305862254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef640b67d0 con 0x7fef640cf2d0 2023-09-12T14:54:35.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.578+0000 7fef60ff9640 1 -- 172.21.15.131:0/3305862254 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef58007290 con 0x7fef640cf2d0 2023-09-12T14:54:35.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.578+0000 7fef6a68b640 1 -- 172.21.15.131:0/3305862254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef640b40a0 con 0x7fef640cf2d0 2023-09-12T14:54:35.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.578+0000 7fef60ff9640 1 -- 172.21.15.131:0/3305862254 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fef580067e0 con 0x7fef640cf2d0 2023-09-12T14:54:35.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.578+0000 7fef60ff9640 1 -- 172.21.15.131:0/3305862254 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fef58011c60 con 0x7fef640cf2d0 2023-09-12T14:54:35.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.578+0000 7fef6a68b640 1 -- 172.21.15.131:0/3305862254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef64073580 con 0x7fef640cf2d0 2023-09-12T14:54:35.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.582+0000 7fef60ff9640 1 -- 172.21.15.131:0/3305862254 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fef58003600 con 0x7fef640cf2d0 2023-09-12T14:54:35.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.634+0000 7fef6a68b640 1 -- 172.21.15.131:0/3305862254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef640affa0 con 0x7fef640cf2d0 2023-09-12T14:54:35.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.634+0000 7fef60ff9640 1 -- 172.21.15.131:0/3305862254 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fef58006c70 con 0x7fef640cf2d0 2023-09-12T14:54:35.639 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:54:35.640 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":10,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:54:35.359818+0000","last_up_change":"0.000000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:54:35.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.634+0000 7fef6a68b640 1 -- 172.21.15.131:0/3305862254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef640cf2d0 msgr2=0x7fef640b3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:35.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.634+0000 7fef6a68b640 1 --2- 172.21.15.131:0/3305862254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef640cf2d0 0x7fef640b3b60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fef5800c500 tx=0x7fef5800c9c0 comp rx=0 tx=0).stop 2023-09-12T14:54:35.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.634+0000 7fef6a68b640 1 -- 172.21.15.131:0/3305862254 shutdown_connections 2023-09-12T14:54:35.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.634+0000 7fef6a68b640 1 --2- 172.21.15.131:0/3305862254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef6415e9f0 0x7fef640cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:35.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.634+0000 7fef6a68b640 1 --2- 172.21.15.131:0/3305862254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef640d1680 0x7fef640cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:35.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.634+0000 7fef6a68b640 1 --2- 172.21.15.131:0/3305862254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef640cf2d0 0x7fef640b3b60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:35.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.634+0000 7fef6a68b640 1 -- 172.21.15.131:0/3305862254 >> 172.21.15.131:0/3305862254 conn(0x7fef6409a200 msgr2=0x7fef640cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:35.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.634+0000 7fef6a68b640 1 -- 172.21.15.131:0/3305862254 shutdown_connections 2023-09-12T14:54:35.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:35.634+0000 7fef6a68b640 1 -- 172.21.15.131:0/3305862254 wait complete. 2023-09-12T14:54:35.655 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-09-12T14:54:35.655 INFO:tasks.ceph:Waiting for OSDs to come up 2023-09-12T14:54:35.748 INFO:tasks.ceph.mgr.y.smithi131.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-09-12T14:54:35.749 INFO:tasks.ceph.mgr.y.smithi131.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-09-12T14:54:35.749 INFO:tasks.ceph.mgr.y.smithi131.stderr: from numpy import show_config as show_numpy_config 2023-09-12T14:54:35.762 INFO:tasks.ceph.osd.0.smithi131.stderr:2023-09-12T14:54:35.758+0000 7fd8576b2640 -1 Falling back to public interface 2023-09-12T14:54:35.793 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-09-12T14:54:35.790+0000 7f19e66bb180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-12T14:54:35.801 INFO:tasks.ceph.osd.1.smithi131.stderr:2023-09-12T14:54:35.798+0000 7f415e9af640 -1 Falling back to public interface 2023-09-12T14:54:35.838 INFO:tasks.ceph.osd.3.smithi131.stderr:2023-09-12T14:54:35.834+0000 7f487c908640 -1 Falling back to public interface 2023-09-12T14:54:35.915 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-09-12T14:54:35.910+0000 7f19e66bb180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-12T14:54:35.926 INFO:tasks.ceph.mgr.x.smithi187.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-09-12T14:54:35.927 INFO:tasks.ceph.mgr.x.smithi187.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-09-12T14:54:35.927 INFO:tasks.ceph.mgr.x.smithi187.stderr: from numpy import show_config as show_numpy_config 2023-09-12T14:54:35.993 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-09-12T14:54:35.990+0000 7f0f107e2180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-12T14:54:36.014 INFO:tasks.ceph.osd.2.smithi131.stderr:2023-09-12T14:54:36.010+0000 7f47bae12640 -1 Falling back to public interface 2023-09-12T14:54:36.037 INFO:tasks.ceph.osd.4.smithi187.stderr:2023-09-12T14:54:36.034+0000 7fa86389c640 -1 Falling back to public interface 2023-09-12T14:54:36.038 INFO:tasks.ceph.osd.6.smithi187.stderr:2023-09-12T14:54:36.034+0000 7f6802c22640 -1 Falling back to public interface 2023-09-12T14:54:36.038 INFO:tasks.ceph.osd.6.smithi187.stderr:2023-09-12T14:54:36.034+0000 7f6801be3640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2023-09-12T14:54:36.056 INFO:tasks.ceph.osd.7.smithi187.stderr:2023-09-12T14:54:36.054+0000 7fbbe52ce640 -1 Falling back to public interface 2023-09-12T14:54:36.064 INFO:tasks.ceph.osd.5.smithi187.stderr:2023-09-12T14:54:36.062+0000 7f410518f640 -1 Falling back to public interface 2023-09-12T14:54:36.071 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-09-12T14:54:36.066+0000 7f19e66bb180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-12T14:54:36.134 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-09-12T14:54:36.130+0000 7f19e66bb180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-12T14:54:36.164 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-09-12T14:54:36.162+0000 7f0f107e2180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-12T14:54:36.257 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-09-12T14:54:36.254+0000 7f19e66bb180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-12T14:54:36.330 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-09-12T14:54:36.326+0000 7f0f107e2180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-12T14:54:36.400 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-09-12T14:54:36.398+0000 7f0f107e2180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-12T14:54:36.462 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-12T14:54:36.535 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-09-12T14:54:36.534+0000 7f0f107e2180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-12T14:54:36.574 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a0e1ee640 1 -- 172.21.15.131:0/1934943658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a0806d860 msgr2=0x7f6a08069240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:36.574 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a0e1ee640 1 --2- 172.21.15.131:0/1934943658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a0806d860 0x7f6a08069240 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f69f0009840 tx=0x7f69f001c740 comp rx=0 tx=0).stop 2023-09-12T14:54:36.575 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a0e1ee640 1 -- 172.21.15.131:0/1934943658 shutdown_connections 2023-09-12T14:54:36.575 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a0e1ee640 1 --2- 172.21.15.131:0/1934943658 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a08071280 0x7f6a080c8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:36.575 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a0e1ee640 1 --2- 172.21.15.131:0/1934943658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a0806d860 0x7f6a08069240 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:36.575 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a0e1ee640 1 -- 172.21.15.131:0/1934943658 >> 172.21.15.131:0/1934943658 conn(0x7f6a0806b310 msgr2=0x7f6a0806b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:36.576 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a0e1ee640 1 -- 172.21.15.131:0/1934943658 shutdown_connections 2023-09-12T14:54:36.576 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a0e1ee640 1 -- 172.21.15.131:0/1934943658 wait complete. 2023-09-12T14:54:36.579 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a0e1ee640 1 Processor -- start 2023-09-12T14:54:36.579 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a0e1ee640 1 -- start start 2023-09-12T14:54:36.579 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a0e1ee640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a0806d860 0x7f6a0814f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:36.580 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a0e1ee640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a08071280 0x7f6a080b0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:36.580 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a0e1ee640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a081567d0 0x7f6a08151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:36.580 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a0e1ee640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0814fdb0 con 0x7f6a081567d0 2023-09-12T14:54:36.581 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a0e1ee640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a08150090 con 0x7f6a08071280 2023-09-12T14:54:36.581 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a0e1ee640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a08150370 con 0x7f6a0806d860 2023-09-12T14:54:36.581 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a07fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a081567d0 0x7f6a08151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:36.581 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a07fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a081567d0 0x7f6a08151440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56326/0 (socket says 172.21.15.131:56326) 2023-09-12T14:54:36.582 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a07fff640 1 -- 172.21.15.131:0/3958003333 learned_addr learned my addr 172.21.15.131:0/3958003333 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:36.582 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f69fffff640 1 --2- 172.21.15.131:0/3958003333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a08071280 0x7f6a080b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:36.582 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a077fe640 1 --2- 172.21.15.131:0/3958003333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a0806d860 0x7f6a0814f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:36.583 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a07fff640 1 -- 172.21.15.131:0/3958003333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a0806d860 msgr2=0x7f6a0814f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:36.583 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a07fff640 1 --2- 172.21.15.131:0/3958003333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a0806d860 0x7f6a0814f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:36.583 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a07fff640 1 -- 172.21.15.131:0/3958003333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a08071280 msgr2=0x7f6a080b0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:36.584 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a07fff640 1 --2- 172.21.15.131:0/3958003333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a08071280 0x7f6a080b0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:36.584 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a07fff640 1 -- 172.21.15.131:0/3958003333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f00094a0 con 0x7f6a081567d0 2023-09-12T14:54:36.584 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a07fff640 1 --2- 172.21.15.131:0/3958003333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a081567d0 0x7f6a08151440 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000eb20 tx=0x7f6a0000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:36.585 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a04ff9640 1 -- 172.21.15.131:0/3958003333 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a00004b30 con 0x7f6a081567d0 2023-09-12T14:54:36.585 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a04ff9640 1 -- 172.21.15.131:0/3958003333 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6a00004cb0 con 0x7f6a081567d0 2023-09-12T14:54:36.585 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a0e1ee640 1 -- 172.21.15.131:0/3958003333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a08150650 con 0x7f6a081567d0 2023-09-12T14:54:36.586 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a04ff9640 1 -- 172.21.15.131:0/3958003333 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a00005280 con 0x7f6a081567d0 2023-09-12T14:54:36.586 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.574+0000 7f6a0e1ee640 1 -- 172.21.15.131:0/3958003333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a0814f9d0 con 0x7f6a081567d0 2023-09-12T14:54:36.586 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.578+0000 7f6a04ff9640 1 -- 172.21.15.131:0/3958003333 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6a000053e0 con 0x7f6a081567d0 2023-09-12T14:54:36.587 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.578+0000 7f6a04ff9640 1 -- 172.21.15.131:0/3958003333 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f6a00014070 con 0x7f6a081567d0 2023-09-12T14:54:36.587 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.578+0000 7f6a0e1ee640 1 -- 172.21.15.131:0/3958003333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a08073580 con 0x7f6a081567d0 2023-09-12T14:54:36.588 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.578+0000 7f6a04ff9640 1 -- 172.21.15.131:0/3958003333 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f6a00004e60 con 0x7f6a081567d0 2023-09-12T14:54:36.636 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.634+0000 7f6a0e1ee640 1 -- 172.21.15.131:0/3958003333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a0815cfe0 con 0x7f6a081567d0 2023-09-12T14:54:36.637 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.634+0000 7f6a04ff9640 1 -- 172.21.15.131:0/3958003333 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f6a00005870 con 0x7f6a081567d0 2023-09-12T14:54:36.637 INFO:teuthology.misc.health.smithi131.stdout: 2023-09-12T14:54:36.637 INFO:teuthology.misc.health.smithi131.stdout:{"epoch":10,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:54:35.359818+0000","last_up_change":"0.000000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:54:36.639 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.638+0000 7f6a0e1ee640 1 -- 172.21.15.131:0/3958003333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a081567d0 msgr2=0x7f6a08151440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:36.639 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.638+0000 7f6a0e1ee640 1 --2- 172.21.15.131:0/3958003333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a081567d0 0x7f6a08151440 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000eb20 tx=0x7f6a0000c2d0 comp rx=0 tx=0).stop 2023-09-12T14:54:36.639 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.638+0000 7f6a0e1ee640 1 -- 172.21.15.131:0/3958003333 shutdown_connections 2023-09-12T14:54:36.639 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.638+0000 7f6a0e1ee640 1 --2- 172.21.15.131:0/3958003333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a081567d0 0x7f6a08151440 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:36.639 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.638+0000 7f6a0e1ee640 1 --2- 172.21.15.131:0/3958003333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a08071280 0x7f6a080b0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:36.640 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.638+0000 7f6a0e1ee640 1 --2- 172.21.15.131:0/3958003333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a0806d860 0x7f6a0814f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:36.640 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.638+0000 7f6a0e1ee640 1 -- 172.21.15.131:0/3958003333 >> 172.21.15.131:0/3958003333 conn(0x7f6a0806b310 msgr2=0x7f6a08152320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:36.640 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.638+0000 7f6a0e1ee640 1 -- 172.21.15.131:0/3958003333 shutdown_connections 2023-09-12T14:54:36.640 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:36.638+0000 7f6a0e1ee640 1 -- 172.21.15.131:0/3958003333 wait complete. 2023-09-12T14:54:36.652 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-09-12T14:54:36.878 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-09-12T14:54:36.878+0000 7f19e66bb180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-12T14:54:36.953 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-09-12T14:54:36.950+0000 7f19e66bb180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-12T14:54:37.122 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-09-12T14:54:37.122+0000 7f19e66bb180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-12T14:54:37.167 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-09-12T14:54:37.166+0000 7f0f107e2180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-12T14:54:37.196 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-09-12T14:54:37.194+0000 7f19e66bb180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-12T14:54:37.243 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-09-12T14:54:37.242+0000 7f0f107e2180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-12T14:54:37.417 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-09-12T14:54:37.410+0000 7f0f107e2180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-12T14:54:37.484 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-09-12T14:54:37.482+0000 7f19e66bb180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-12T14:54:37.487 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-09-12T14:54:37.482+0000 7f0f107e2180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-12T14:54:37.543 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-09-12T14:54:37.542+0000 7f19e66bb180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-12T14:54:37.780 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-09-12T14:54:37.778+0000 7f0f107e2180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-12T14:54:37.839 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-09-12T14:54:37.834+0000 7f0f107e2180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-12T14:54:38.582 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-09-12T14:54:38.582+0000 7f198df32640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-09-12T14:54:38.585 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-09-12T14:54:38.586+0000 7f198df32640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-09-12T14:54:39.674 INFO:tasks.ceph.osd.0.smithi131.stderr:2023-09-12T14:54:39.674+0000 7fd8576b2640 -1 osd.0 0 log_to_monitors true 2023-09-12T14:54:39.685 INFO:tasks.ceph.osd.1.smithi131.stderr:2023-09-12T14:54:39.682+0000 7f415e9af640 -1 osd.1 0 log_to_monitors true 2023-09-12T14:54:39.707 INFO:tasks.ceph.osd.3.smithi131.stderr:2023-09-12T14:54:39.706+0000 7f487c908640 -1 osd.3 0 log_to_monitors true 2023-09-12T14:54:39.807 INFO:tasks.ceph.osd.2.smithi131.stderr:2023-09-12T14:54:39.806+0000 7f47bae12640 -1 osd.2 0 log_to_monitors true 2023-09-12T14:54:39.910 INFO:tasks.ceph.osd.6.smithi187.stderr:2023-09-12T14:54:39.906+0000 7f6802c22640 -1 osd.6 0 log_to_monitors true 2023-09-12T14:54:39.953 INFO:tasks.ceph.osd.7.smithi187.stderr:2023-09-12T14:54:39.950+0000 7fbbe52ce640 -1 osd.7 0 log_to_monitors true 2023-09-12T14:54:39.969 INFO:tasks.ceph.osd.5.smithi187.stderr:2023-09-12T14:54:39.966+0000 7f410518f640 -1 osd.5 0 log_to_monitors true 2023-09-12T14:54:39.997 INFO:tasks.ceph.osd.4.smithi187.stderr:2023-09-12T14:54:39.994+0000 7fa86389c640 -1 osd.4 0 log_to_monitors true 2023-09-12T14:54:43.458 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-12T14:54:43.660 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.658+0000 7f019adc9640 1 -- 172.21.15.131:0/4276230187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f019406fdd0 msgr2=0x7f0194070240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:43.661 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.658+0000 7f019adc9640 1 --2- 172.21.15.131:0/4276230187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f019406fdd0 0x7f0194070240 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f019406e980 tx=0x7f018c01cb30 comp rx=0 tx=0).stop 2023-09-12T14:54:43.661 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.658+0000 7f019adc9640 1 -- 172.21.15.131:0/4276230187 shutdown_connections 2023-09-12T14:54:43.661 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.658+0000 7f019adc9640 1 --2- 172.21.15.131:0/4276230187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f019406fdd0 0x7f0194070240 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:43.662 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.658+0000 7f019adc9640 1 --2- 172.21.15.131:0/4276230187 >> v2:172.21.15.131:3300/0 conn(0x7f01941556e0 0x7f0194158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:43.662 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.658+0000 7f019adc9640 1 -- 172.21.15.131:0/4276230187 >> 172.21.15.131:0/4276230187 conn(0x7f019406a950 msgr2=0x7f01940af5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:43.662 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.658+0000 7f019adc9640 1 -- 172.21.15.131:0/4276230187 shutdown_connections 2023-09-12T14:54:43.662 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.658+0000 7f019adc9640 1 -- 172.21.15.131:0/4276230187 wait complete. 2023-09-12T14:54:43.665 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.662+0000 7f019adc9640 1 Processor -- start 2023-09-12T14:54:43.665 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.662+0000 7f019adc9640 1 -- start start 2023-09-12T14:54:43.666 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.662+0000 7f019adc9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f019406fdd0 0x7f01940cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:43.666 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.662+0000 7f019adc9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01940b2100 0x7f0194150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:43.666 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.662+0000 7f019adc9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01941556e0 0x7f019415ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:43.666 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.662+0000 7f019adc9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019415a7e0 con 0x7f01940b2100 2023-09-12T14:54:43.667 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.662+0000 7f019adc9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01940cbf90 con 0x7f019406fdd0 2023-09-12T14:54:43.667 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.662+0000 7f0198b65640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f019406fdd0 0x7f01940cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:43.667 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.662+0000 7f0193fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01940b2100 0x7f0194150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:43.667 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.662+0000 7f019adc9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0194159eb0 con 0x7f01941556e0 2023-09-12T14:54:43.668 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.662+0000 7f0193fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01940b2100 0x7f0194150c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52738/0 (socket says 172.21.15.131:52738) 2023-09-12T14:54:43.668 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.662+0000 7f0193fff640 1 -- 172.21.15.131:0/2445075568 learned_addr learned my addr 172.21.15.131:0/2445075568 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:43.668 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.662+0000 7f0199366640 1 --2- 172.21.15.131:0/2445075568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01941556e0 0x7f019415ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:43.668 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.662+0000 7f0199366640 1 -- 172.21.15.131:0/2445075568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f019406fdd0 msgr2=0x7f01940cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:43.668 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.662+0000 7f0199366640 1 --2- 172.21.15.131:0/2445075568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f019406fdd0 0x7f01940cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:43.669 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.662+0000 7f0199366640 1 -- 172.21.15.131:0/2445075568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01940b2100 msgr2=0x7f0194150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:43.669 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.662+0000 7f0199366640 1 --2- 172.21.15.131:0/2445075568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01940b2100 0x7f0194150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:43.669 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.662+0000 7f0199366640 1 -- 172.21.15.131:0/2445075568 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f018c009820 con 0x7f01941556e0 2023-09-12T14:54:43.669 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.662+0000 7f0199366640 1 --2- 172.21.15.131:0/2445075568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01941556e0 0x7f019415ac70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f018c0ac000 tx=0x7f018c0a5b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:43.669 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.662+0000 7f01917fa640 1 -- 172.21.15.131:0/2445075568 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c003e10 con 0x7f01941556e0 2023-09-12T14:54:43.670 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.662+0000 7f01917fa640 1 -- 172.21.15.131:0/2445075568 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f018c004130 con 0x7f01941556e0 2023-09-12T14:54:43.670 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.662+0000 7f01917fa640 1 -- 172.21.15.131:0/2445075568 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c004540 con 0x7f01941556e0 2023-09-12T14:54:43.670 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.662+0000 7f019adc9640 1 -- 172.21.15.131:0/2445075568 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f019415a190 con 0x7f01941556e0 2023-09-12T14:54:43.671 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.666+0000 7f019adc9640 1 -- 172.21.15.131:0/2445075568 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01940cbb60 con 0x7f01941556e0 2023-09-12T14:54:43.671 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.666+0000 7f019adc9640 1 -- 172.21.15.131:0/2445075568 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f019415a950 con 0x7f01941556e0 2023-09-12T14:54:43.672 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.666+0000 7f01917fa640 1 -- 172.21.15.131:0/2445075568 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 4) v1 ==== 85298+0+0 (secure 0 0 0) 0x7f018c0b4040 con 0x7f01941556e0 2023-09-12T14:54:43.672 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.666+0000 7f01917fa640 1 --2- 172.21.15.131:0/2445075568 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0178065ac0 0x7f0178067f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:43.672 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.666+0000 7f01917fa640 1 -- 172.21.15.131:0/2445075568 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f018c11d850 con 0x7f01941556e0 2023-09-12T14:54:43.672 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.666+0000 7f0198b65640 1 --2- 172.21.15.131:0/2445075568 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0178065ac0 0x7f0178067f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:43.672 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.666+0000 7f0198b65640 1 --2- 172.21.15.131:0/2445075568 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0178065ac0 0x7f0178067f80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f018800b9a0 tx=0x7f0188005c30 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:43.675 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.674+0000 7f01917fa640 1 -- 172.21.15.131:0/2445075568 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f018c0ec080 con 0x7f01941556e0 2023-09-12T14:54:43.785 INFO:tasks.ceph.osd.5.smithi187.stderr:2023-09-12T14:54:43.778+0000 7f4100914640 -1 osd.5 0 waiting for initial osdmap 2023-09-12T14:54:43.819 INFO:tasks.ceph.osd.5.smithi187.stderr:2023-09-12T14:54:43.818+0000 7f40f8f5b640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-12T14:54:43.858 INFO:tasks.ceph.osd.7.smithi187.stderr:2023-09-12T14:54:43.854+0000 7fbbe0a53640 -1 osd.7 0 waiting for initial osdmap 2023-09-12T14:54:43.873 INFO:tasks.ceph.osd.4.smithi187.stderr:2023-09-12T14:54:43.870+0000 7fa85f021640 -1 osd.4 0 waiting for initial osdmap 2023-09-12T14:54:43.875 INFO:tasks.ceph.osd.7.smithi187.stderr:2023-09-12T14:54:43.874+0000 7fbbd909a640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-12T14:54:43.891 INFO:tasks.ceph.osd.6.smithi187.stderr:2023-09-12T14:54:43.890+0000 7f67ffbdf640 -1 osd.6 0 waiting for initial osdmap 2023-09-12T14:54:43.899 INFO:tasks.ceph.osd.4.smithi187.stderr:2023-09-12T14:54:43.898+0000 7fa857668640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-12T14:54:43.935 INFO:tasks.ceph.osd.6.smithi187.stderr:2023-09-12T14:54:43.934+0000 7f67f69ee640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-12T14:54:43.953 INFO:tasks.ceph.osd.2.smithi131.stderr:2023-09-12T14:54:43.950+0000 7f47b6597640 -1 osd.2 0 waiting for initial osdmap 2023-09-12T14:54:43.972 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.970+0000 7f019adc9640 1 -- 172.21.15.131:0/2445075568 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0194159ff0 con 0x7f01941556e0 2023-09-12T14:54:43.973 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.970+0000 7f01917fa640 1 -- 172.21.15.131:0/2445075568 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f0194159ff0 con 0x7f01941556e0 2023-09-12T14:54:43.973 INFO:teuthology.misc.health.smithi131.stdout: 2023-09-12T14:54:43.973 INFO:teuthology.misc.health.smithi131.stdout:{"epoch":12,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:54:41.590080+0000","last_up_change":"0.000000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:54:43.976 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.974+0000 7f019adc9640 1 -- 172.21.15.131:0/2445075568 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0178065ac0 msgr2=0x7f0178067f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:43.976 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.974+0000 7f019adc9640 1 --2- 172.21.15.131:0/2445075568 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0178065ac0 0x7f0178067f80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f018800b9a0 tx=0x7f0188005c30 comp rx=0 tx=0).stop 2023-09-12T14:54:43.976 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.974+0000 7f019adc9640 1 -- 172.21.15.131:0/2445075568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01941556e0 msgr2=0x7f019415ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:43.976 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.974+0000 7f019adc9640 1 --2- 172.21.15.131:0/2445075568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01941556e0 0x7f019415ac70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f018c0ac000 tx=0x7f018c0a5b60 comp rx=0 tx=0).stop 2023-09-12T14:54:43.977 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.974+0000 7f019adc9640 1 -- 172.21.15.131:0/2445075568 shutdown_connections 2023-09-12T14:54:43.977 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.974+0000 7f019adc9640 1 --2- 172.21.15.131:0/2445075568 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0178065ac0 0x7f0178067f80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:43.977 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.974+0000 7f019adc9640 1 --2- 172.21.15.131:0/2445075568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01941556e0 0x7f019415ac70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:43.977 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.974+0000 7f019adc9640 1 --2- 172.21.15.131:0/2445075568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01940b2100 0x7f0194150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:43.977 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.974+0000 7f019adc9640 1 --2- 172.21.15.131:0/2445075568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f019406fdd0 0x7f01940cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:43.978 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.974+0000 7f019adc9640 1 -- 172.21.15.131:0/2445075568 >> 172.21.15.131:0/2445075568 conn(0x7f019406a950 msgr2=0x7f01940c9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:43.978 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.974+0000 7f019adc9640 1 -- 172.21.15.131:0/2445075568 shutdown_connections 2023-09-12T14:54:43.978 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:43.974+0000 7f019adc9640 1 -- 172.21.15.131:0/2445075568 wait complete. 2023-09-12T14:54:43.997 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-09-12T14:54:43.998 INFO:tasks.ceph.osd.2.smithi131.stderr:2023-09-12T14:54:43.998+0000 7f47aebde640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-12T14:54:44.057 INFO:tasks.ceph.osd.1.smithi131.stderr:2023-09-12T14:54:44.058+0000 7f415a134640 -1 osd.1 0 waiting for initial osdmap 2023-09-12T14:54:44.073 INFO:tasks.ceph.osd.3.smithi131.stderr:2023-09-12T14:54:44.074+0000 7f487808d640 -1 osd.3 0 waiting for initial osdmap 2023-09-12T14:54:44.074 INFO:tasks.ceph.osd.0.smithi131.stderr:2023-09-12T14:54:44.074+0000 7fd852e37640 -1 osd.0 0 waiting for initial osdmap 2023-09-12T14:54:44.093 INFO:tasks.ceph.osd.1.smithi131.stderr:2023-09-12T14:54:44.094+0000 7f415277b640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-12T14:54:44.109 INFO:tasks.ceph.osd.3.smithi131.stderr:2023-09-12T14:54:44.110+0000 7f48706d4640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-12T14:54:44.113 INFO:tasks.ceph.osd.0.smithi131.stderr:2023-09-12T14:54:44.114+0000 7fd84b47e640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-12T14:54:50.805 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-12T14:54:50.931 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe8383a8640 1 -- 172.21.15.131:0/4041631527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8300abb40 msgr2=0x7fe83006b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:50.931 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe8383a8640 1 --2- 172.21.15.131:0/4041631527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8300abb40 0x7fe83006b4a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe8300c8d20 tx=0x7fe82001c830 comp rx=0 tx=0).stop 2023-09-12T14:54:50.931 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe8383a8640 1 -- 172.21.15.131:0/4041631527 shutdown_connections 2023-09-12T14:54:50.932 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe8383a8640 1 --2- 172.21.15.131:0/4041631527 >> v2:172.21.15.187:3300/0 conn(0x7fe8300c9370 0x7fe8300c9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:50.932 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe8383a8640 1 --2- 172.21.15.131:0/4041631527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8300abb40 0x7fe83006b4a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:50.932 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe8383a8640 1 -- 172.21.15.131:0/4041631527 >> 172.21.15.131:0/4041631527 conn(0x7fe830069840 msgr2=0x7fe8300cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:50.932 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe8383a8640 1 -- 172.21.15.131:0/4041631527 shutdown_connections 2023-09-12T14:54:50.933 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe8383a8640 1 -- 172.21.15.131:0/4041631527 wait complete. 2023-09-12T14:54:50.935 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe8383a8640 1 Processor -- start 2023-09-12T14:54:50.935 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe8383a8640 1 -- start start 2023-09-12T14:54:50.935 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe8383a8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8300abb40 0x7fe8301469c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:50.935 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe8383a8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8300c9370 0x7fe83014eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:50.936 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe8383a8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe830156930 0x7fe830154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:50.936 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe8383a8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83006ff90 con 0x7fe830156930 2023-09-12T14:54:50.936 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe8383a8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8300caaa0 con 0x7fe8300abb40 2023-09-12T14:54:50.936 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe8383a8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83014f550 con 0x7fe8300c9370 2023-09-12T14:54:50.936 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe836945640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe830156930 0x7fe830154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:50.937 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe836945640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe830156930 0x7fe830154d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56036/0 (socket says 172.21.15.131:56036) 2023-09-12T14:54:50.937 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe836945640 1 -- 172.21.15.131:0/231061855 learned_addr learned my addr 172.21.15.131:0/231061855 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:50.937 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe836144640 1 --2- 172.21.15.131:0/231061855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8300abb40 0x7fe8301469c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:50.937 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe835943640 1 --2- 172.21.15.131:0/231061855 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8300c9370 0x7fe83014eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:50.937 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe836945640 1 -- 172.21.15.131:0/231061855 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8300c9370 msgr2=0x7fe83014eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:50.938 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe836945640 1 --2- 172.21.15.131:0/231061855 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8300c9370 0x7fe83014eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:50.938 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe836945640 1 -- 172.21.15.131:0/231061855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8300abb40 msgr2=0x7fe8301469c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:50.938 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe836945640 1 --2- 172.21.15.131:0/231061855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8300abb40 0x7fe8301469c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:50.938 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe836945640 1 -- 172.21.15.131:0/231061855 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe820009560 con 0x7fe830156930 2023-09-12T14:54:50.938 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe836144640 1 --2- 172.21.15.131:0/231061855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8300abb40 0x7fe8301469c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:54:50.939 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe836945640 1 --2- 172.21.15.131:0/231061855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe830156930 0x7fe830154d60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe82800cce0 tx=0x7fe82800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:50.939 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe826ffd640 1 -- 172.21.15.131:0/231061855 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82800f450 con 0x7fe830156930 2023-09-12T14:54:50.939 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.929+0000 7fe826ffd640 1 -- 172.21.15.131:0/231061855 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe82800f6c0 con 0x7fe830156930 2023-09-12T14:54:50.939 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.933+0000 7fe826ffd640 1 -- 172.21.15.131:0/231061855 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82800fad0 con 0x7fe830156930 2023-09-12T14:54:50.939 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.933+0000 7fe8383a8640 1 -- 172.21.15.131:0/231061855 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83006f9d0 con 0x7fe830156930 2023-09-12T14:54:50.940 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.933+0000 7fe8383a8640 1 -- 172.21.15.131:0/231061855 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8301477b0 con 0x7fe830156930 2023-09-12T14:54:50.940 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.933+0000 7fe8383a8640 1 -- 172.21.15.131:0/231061855 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8300ca800 con 0x7fe830156930 2023-09-12T14:54:50.941 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.933+0000 7fe826ffd640 1 -- 172.21.15.131:0/231061855 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fe82800fc30 con 0x7fe830156930 2023-09-12T14:54:50.941 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.933+0000 7fe826ffd640 1 --2- 172.21.15.131:0/231061855 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe814065b20 0x7fe814067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:50.941 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.933+0000 7fe826ffd640 1 -- 172.21.15.131:0/231061855 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe828083b40 con 0x7fe830156930 2023-09-12T14:54:50.941 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.933+0000 7fe836144640 1 --2- 172.21.15.131:0/231061855 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe814065b20 0x7fe814067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:50.941 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.933+0000 7fe836144640 1 --2- 172.21.15.131:0/231061855 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe814065b20 0x7fe814067fe0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009890 tx=0x7fe82c008040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:50.942 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:50.937+0000 7fe826ffd640 1 -- 172.21.15.131:0/231061855 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fe828051850 con 0x7fe830156930 2023-09-12T14:54:51.096 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:51.093+0000 7fe8383a8640 1 -- 172.21.15.131:0/231061855 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8300cabe0 con 0x7fe830156930 2023-09-12T14:54:51.097 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:51.097+0000 7fe826ffd640 1 -- 172.21.15.131:0/231061855 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12886 (secure 0 0 0) 0x7fe8300cabe0 con 0x7fe830156930 2023-09-12T14:54:51.098 INFO:teuthology.misc.health.smithi131.stdout: 2023-09-12T14:54:51.098 INFO:teuthology.misc.health.smithi131.stdout:{"epoch":16,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:54:47.872696+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:54:51.100 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:51.097+0000 7fe8383a8640 1 -- 172.21.15.131:0/231061855 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe814065b20 msgr2=0x7fe814067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:51.100 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:51.097+0000 7fe8383a8640 1 --2- 172.21.15.131:0/231061855 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe814065b20 0x7fe814067fe0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009890 tx=0x7fe82c008040 comp rx=0 tx=0).stop 2023-09-12T14:54:51.101 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:51.097+0000 7fe8383a8640 1 -- 172.21.15.131:0/231061855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe830156930 msgr2=0x7fe830154d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:51.101 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:51.097+0000 7fe8383a8640 1 --2- 172.21.15.131:0/231061855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe830156930 0x7fe830154d60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe82800cce0 tx=0x7fe82800a620 comp rx=0 tx=0).stop 2023-09-12T14:54:51.101 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:51.097+0000 7fe8383a8640 1 -- 172.21.15.131:0/231061855 shutdown_connections 2023-09-12T14:54:51.101 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:51.097+0000 7fe8383a8640 1 --2- 172.21.15.131:0/231061855 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe814065b20 0x7fe814067fe0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.101 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:51.097+0000 7fe8383a8640 1 --2- 172.21.15.131:0/231061855 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8300c9370 0x7fe83014eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.102 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:51.097+0000 7fe8383a8640 1 --2- 172.21.15.131:0/231061855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe830156930 0x7fe830154d60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.102 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:51.097+0000 7fe8383a8640 1 --2- 172.21.15.131:0/231061855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8300abb40 0x7fe8301469c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.102 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:51.097+0000 7fe8383a8640 1 -- 172.21.15.131:0/231061855 >> 172.21.15.131:0/231061855 conn(0x7fe830069840 msgr2=0x7fe83014f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:51.102 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:51.097+0000 7fe8383a8640 1 -- 172.21.15.131:0/231061855 shutdown_connections 2023-09-12T14:54:51.103 INFO:teuthology.misc.health.smithi131.stderr:2023-09-12T14:54:51.097+0000 7fe8383a8640 1 -- 172.21.15.131:0/231061855 wait complete. 2023-09-12T14:54:51.116 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-09-12T14:54:51.116 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-09-12T14:54:51.116 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-12T14:54:51.117 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-12T14:54:51.119 INFO:tasks.ceph.mds.a:Started 2023-09-12T14:54:51.120 INFO:tasks.ceph.mds.c:Restarting daemon 2023-09-12T14:54:51.120 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-09-12T14:54:51.122 INFO:tasks.ceph.mds.c:Started 2023-09-12T14:54:51.122 INFO:tasks.ceph.mds.b:Restarting daemon 2023-09-12T14:54:51.123 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-09-12T14:54:51.126 INFO:tasks.ceph.mds.b:Started 2023-09-12T14:54:51.126 INFO:tasks.ceph.mds.d:Restarting daemon 2023-09-12T14:54:51.126 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-09-12T14:54:51.129 INFO:tasks.ceph.mds.d:Started 2023-09-12T14:54:51.129 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-09-12T14:54:51.130 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:54:51.189 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2023-09-12T14:54:51.190 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2023-09-12T14:54:51.204 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2023-09-12T14:54:51.208 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2023-09-12T14:54:51.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f0090f60640 1 -- 172.21.15.131:0/2600118113 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f008c06a250 msgr2=0x7f008c0cbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:51.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f0090f60640 1 --2- 172.21.15.131:0/2600118113 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f008c06a250 0x7f008c0cbb40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0084009bc0 tx=0x7f008401cb80 comp rx=0 tx=0).stop 2023-09-12T14:54:51.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f0090f60640 1 -- 172.21.15.131:0/2600118113 shutdown_connections 2023-09-12T14:54:51.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f0090f60640 1 --2- 172.21.15.131:0/2600118113 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f008c06a250 0x7f008c0cbb40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f0090f60640 1 --2- 172.21.15.131:0/2600118113 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f008c06bb70 0x7f008c070010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f0090f60640 1 -- 172.21.15.131:0/2600118113 >> 172.21.15.131:0/2600118113 conn(0x7f008c0cd150 msgr2=0x7f008c0cd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:51.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f0090f60640 1 -- 172.21.15.131:0/2600118113 shutdown_connections 2023-09-12T14:54:51.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f0090f60640 1 -- 172.21.15.131:0/2600118113 wait complete. 2023-09-12T14:54:51.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f0090f60640 1 Processor -- start 2023-09-12T14:54:51.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f0090f60640 1 -- start start 2023-09-12T14:54:51.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f0090f60640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f008c06a250 0x7f008c14fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:51.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f0090f60640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f008c070550 0x7f008c14ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:51.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f0090f60640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f008c14d410 0x7f008c0c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:51.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f0090f60640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c1637e0 con 0x7f008c14d410 2023-09-12T14:54:51.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f0090f60640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c154180 con 0x7f008c070550 2023-09-12T14:54:51.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f0090f60640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008c154460 con 0x7f008c06a250 2023-09-12T14:54:51.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f008ad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f008c14d410 0x7f008c0c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:51.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f008ad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f008c14d410 0x7f008c0c9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56086/0 (socket says 172.21.15.131:56086) 2023-09-12T14:54:51.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f008ad9d640 1 -- 172.21.15.131:0/2543537580 learned_addr learned my addr 172.21.15.131:0/2543537580 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:51.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f008a59c640 1 --2- 172.21.15.131:0/2543537580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f008c06a250 0x7f008c14fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:51.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f0083fff640 1 --2- 172.21.15.131:0/2543537580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f008c070550 0x7f008c14ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:51.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f008ad9d640 1 -- 172.21.15.131:0/2543537580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f008c06a250 msgr2=0x7f008c14fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:51.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f008ad9d640 1 --2- 172.21.15.131:0/2543537580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f008c06a250 0x7f008c14fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f008ad9d640 1 -- 172.21.15.131:0/2543537580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f008c070550 msgr2=0x7f008c14ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:51.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f008ad9d640 1 --2- 172.21.15.131:0/2543537580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f008c070550 0x7f008c14ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f008ad9d640 1 -- 172.21.15.131:0/2543537580 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0084009820 con 0x7f008c14d410 2023-09-12T14:54:51.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f008a59c640 1 --2- 172.21.15.131:0/2543537580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f008c06a250 0x7f008c14fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:54:51.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f0083fff640 1 --2- 172.21.15.131:0/2543537580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f008c070550 0x7f008c14ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:54:51.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f008ad9d640 1 --2- 172.21.15.131:0/2543537580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f008c14d410 0x7f008c0c9b40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0084009b10 tx=0x7f00840a5bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:51.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f0082ffd640 1 -- 172.21.15.131:0/2543537580 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00840a5ea0 con 0x7f008c14d410 2023-09-12T14:54:51.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f0082ffd640 1 -- 172.21.15.131:0/2543537580 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0084004e00 con 0x7f008c14d410 2023-09-12T14:54:51.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f0082ffd640 1 -- 172.21.15.131:0/2543537580 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0084004200 con 0x7f008c14d410 2023-09-12T14:54:51.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f0090f60640 1 -- 172.21.15.131:0/2543537580 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f008c154740 con 0x7f008c14d410 2023-09-12T14:54:51.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f0090f60640 1 -- 172.21.15.131:0/2543537580 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f008c154d00 con 0x7f008c14d410 2023-09-12T14:54:51.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.257+0000 7f0090f60640 1 -- 172.21.15.131:0/2543537580 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f008c154ef0 con 0x7f008c14d410 2023-09-12T14:54:51.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.261+0000 7f0082ffd640 1 -- 172.21.15.131:0/2543537580 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f00840b4040 con 0x7f008c14d410 2023-09-12T14:54:51.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.261+0000 7f0082ffd640 1 --2- 172.21.15.131:0/2543537580 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f006c065b20 0x7f006c067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:51.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.261+0000 7f008a59c640 1 --2- 172.21.15.131:0/2543537580 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f006c065b20 0x7f006c067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:51.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.261+0000 7f0082ffd640 1 -- 172.21.15.131:0/2543537580 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f008411cf90 con 0x7f008c14d410 2023-09-12T14:54:51.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.261+0000 7f008a59c640 1 --2- 172.21.15.131:0/2543537580 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f006c065b20 0x7f006c067fe0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0074005fd0 tx=0x7f007400a5e0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:51.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.265+0000 7f0082ffd640 1 -- 172.21.15.131:0/2543537580 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f00840eac30 con 0x7f008c14d410 2023-09-12T14:54:51.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.417+0000 7f0090f60640 1 -- 172.21.15.131:0/2543537580 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f008c0ccd20 con 0x7f008c14d410 2023-09-12T14:54:51.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.417+0000 7f0082ffd640 1 -- 172.21.15.131:0/2543537580 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12886 (secure 0 0 0) 0x7f00840b2020 con 0x7f008c14d410 2023-09-12T14:54:51.422 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:54:51.422 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":16,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:54:47.872696+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:54:51.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.421+0000 7f0090f60640 1 -- 172.21.15.131:0/2543537580 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f006c065b20 msgr2=0x7f006c067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:51.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.421+0000 7f0090f60640 1 --2- 172.21.15.131:0/2543537580 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f006c065b20 0x7f006c067fe0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0074005fd0 tx=0x7f007400a5e0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.421+0000 7f0090f60640 1 -- 172.21.15.131:0/2543537580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f008c14d410 msgr2=0x7f008c0c9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:51.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.421+0000 7f0090f60640 1 --2- 172.21.15.131:0/2543537580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f008c14d410 0x7f008c0c9b40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0084009b10 tx=0x7f00840a5bd0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.421+0000 7f0090f60640 1 -- 172.21.15.131:0/2543537580 shutdown_connections 2023-09-12T14:54:51.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.421+0000 7f0090f60640 1 --2- 172.21.15.131:0/2543537580 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f006c065b20 0x7f006c067fe0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.421+0000 7f0090f60640 1 --2- 172.21.15.131:0/2543537580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f008c14d410 0x7f008c0c9b40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.421+0000 7f0090f60640 1 --2- 172.21.15.131:0/2543537580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f008c070550 0x7f008c14ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.421+0000 7f0090f60640 1 --2- 172.21.15.131:0/2543537580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f008c06a250 0x7f008c14fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.421+0000 7f0090f60640 1 -- 172.21.15.131:0/2543537580 >> 172.21.15.131:0/2543537580 conn(0x7f008c0cd150 msgr2=0x7f008c0ae3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:51.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.421+0000 7f0090f60640 1 -- 172.21.15.131:0/2543537580 shutdown_connections 2023-09-12T14:54:51.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.421+0000 7f0090f60640 1 -- 172.21.15.131:0/2543537580 wait complete. 2023-09-12T14:54:51.440 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T14:54:45.654004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-12T14:54:51.441 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T14:54:51.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45d4960640 1 -- 172.21.15.131:0/4147299670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d006a350 msgr2=0x7f45d006a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:51.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45d4960640 1 --2- 172.21.15.131:0/4147299670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d006a350 0x7f45d006a730 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f45c0009900 tx=0x7f45c001c780 comp rx=0 tx=0).stop 2023-09-12T14:54:51.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45d4960640 1 -- 172.21.15.131:0/4147299670 shutdown_connections 2023-09-12T14:54:51.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45d4960640 1 --2- 172.21.15.131:0/4147299670 >> v2:172.21.15.187:3300/0 conn(0x7f45d0157d90 0x7f45d006fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45d4960640 1 --2- 172.21.15.131:0/4147299670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d006a350 0x7f45d006a730 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45d4960640 1 -- 172.21.15.131:0/4147299670 >> 172.21.15.131:0/4147299670 conn(0x7f45d006b3a0 msgr2=0x7f45d006b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:51.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45d4960640 1 -- 172.21.15.131:0/4147299670 shutdown_connections 2023-09-12T14:54:51.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45d4960640 1 -- 172.21.15.131:0/4147299670 wait complete. 2023-09-12T14:54:51.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45d4960640 1 Processor -- start 2023-09-12T14:54:51.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45d4960640 1 -- start start 2023-09-12T14:54:51.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45d4960640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45d006a350 0x7f45d0149cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:51.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45d4960640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d006ffa0 0x7f45d01f4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:51.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45d4960640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45d00ac150 0x7f45d015ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:51.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45d4960640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d014a1f0 con 0x7f45d006ffa0 2023-09-12T14:54:51.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45d4960640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d01f4860 con 0x7f45d006a350 2023-09-12T14:54:51.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45d4960640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d01590c0 con 0x7f45d00ac150 2023-09-12T14:54:51.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45cdd9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d006ffa0 0x7f45d01f4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:51.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45cdd9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d006ffa0 0x7f45d01f4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56102/0 (socket says 172.21.15.131:56102) 2023-09-12T14:54:51.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45cdd9b640 1 -- 172.21.15.131:0/740586382 learned_addr learned my addr 172.21.15.131:0/740586382 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:51.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45ced9d640 1 --2- 172.21.15.131:0/740586382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45d00ac150 0x7f45d015ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:51.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45ce59c640 1 --2- 172.21.15.131:0/740586382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45d006a350 0x7f45d0149cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:51.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45cdd9b640 1 -- 172.21.15.131:0/740586382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45d00ac150 msgr2=0x7f45d015ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:51.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45cdd9b640 1 --2- 172.21.15.131:0/740586382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45d00ac150 0x7f45d015ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45cdd9b640 1 -- 172.21.15.131:0/740586382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45d006a350 msgr2=0x7f45d0149cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:51.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45cdd9b640 1 --2- 172.21.15.131:0/740586382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45d006a350 0x7f45d0149cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45cdd9b640 1 -- 172.21.15.131:0/740586382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c0009560 con 0x7f45d006ffa0 2023-09-12T14:54:51.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45ce59c640 1 --2- 172.21.15.131:0/740586382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45d006a350 0x7f45d0149cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:54:51.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45cdd9b640 1 --2- 172.21.15.131:0/740586382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d006ffa0 0x7f45d01f4320 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f45c400ca20 tx=0x7f45c400cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:51.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45beffd640 1 -- 172.21.15.131:0/740586382 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c4007c70 con 0x7f45d006ffa0 2023-09-12T14:54:51.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45beffd640 1 -- 172.21.15.131:0/740586382 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f45c400f450 con 0x7f45d006ffa0 2023-09-12T14:54:51.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45d4960640 1 -- 172.21.15.131:0/740586382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d0159680 con 0x7f45d006ffa0 2023-09-12T14:54:51.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45beffd640 1 -- 172.21.15.131:0/740586382 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c400f860 con 0x7f45d006ffa0 2023-09-12T14:54:51.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.573+0000 7f45d4960640 1 -- 172.21.15.131:0/740586382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d015f530 con 0x7f45d006ffa0 2023-09-12T14:54:51.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.577+0000 7f45d4960640 1 -- 172.21.15.131:0/740586382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d0159810 con 0x7f45d006ffa0 2023-09-12T14:54:51.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.577+0000 7f45beffd640 1 -- 172.21.15.131:0/740586382 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f45c400f9c0 con 0x7f45d006ffa0 2023-09-12T14:54:51.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.577+0000 7f45beffd640 1 --2- 172.21.15.131:0/740586382 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f45b0065b20 0x7f45b0067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:51.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.577+0000 7f45ce59c640 1 --2- 172.21.15.131:0/740586382 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f45b0065b20 0x7f45b0067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:51.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.577+0000 7f45beffd640 1 -- 172.21.15.131:0/740586382 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f45c40837a0 con 0x7f45d006ffa0 2023-09-12T14:54:51.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.577+0000 7f45ce59c640 1 --2- 172.21.15.131:0/740586382 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f45b0065b20 0x7f45b0067fe0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f45c00af000 tx=0x7f45c001afc0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:51.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.581+0000 7f45beffd640 1 -- 172.21.15.131:0/740586382 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f45c4051440 con 0x7f45d006ffa0 2023-09-12T14:54:51.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.733+0000 7f45d4960640 1 -- 172.21.15.131:0/740586382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f45d015f750 con 0x7f45d006ffa0 2023-09-12T14:54:51.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.733+0000 7f45beffd640 1 -- 172.21.15.131:0/740586382 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f45c4054a60 con 0x7f45d006ffa0 2023-09-12T14:54:51.737 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-09-12T14:54:51.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.737+0000 7f45d4960640 1 -- 172.21.15.131:0/740586382 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f45b0065b20 msgr2=0x7f45b0067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:51.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.737+0000 7f45d4960640 1 --2- 172.21.15.131:0/740586382 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f45b0065b20 0x7f45b0067fe0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f45c00af000 tx=0x7f45c001afc0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.737+0000 7f45d4960640 1 -- 172.21.15.131:0/740586382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d006ffa0 msgr2=0x7f45d01f4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:51.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.737+0000 7f45d4960640 1 --2- 172.21.15.131:0/740586382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d006ffa0 0x7f45d01f4320 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f45c400ca20 tx=0x7f45c400cee0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.737+0000 7f45d4960640 1 -- 172.21.15.131:0/740586382 shutdown_connections 2023-09-12T14:54:51.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.737+0000 7f45d4960640 1 --2- 172.21.15.131:0/740586382 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f45b0065b20 0x7f45b0067fe0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.737+0000 7f45d4960640 1 --2- 172.21.15.131:0/740586382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45d00ac150 0x7f45d015ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.737+0000 7f45d4960640 1 --2- 172.21.15.131:0/740586382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d006ffa0 0x7f45d01f4320 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.737+0000 7f45d4960640 1 --2- 172.21.15.131:0/740586382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45d006a350 0x7f45d0149cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.737+0000 7f45d4960640 1 -- 172.21.15.131:0/740586382 >> 172.21.15.131:0/740586382 conn(0x7f45d006b3a0 msgr2=0x7f45d00cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:51.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.737+0000 7f45d4960640 1 -- 172.21.15.131:0/740586382 shutdown_connections 2023-09-12T14:54:51.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.737+0000 7f45d4960640 1 -- 172.21.15.131:0/740586382 wait complete. 2023-09-12T14:54:51.756 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T14:54:51.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.881+0000 7fb477f72640 1 -- 172.21.15.131:0/3305551707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb47015bb80 msgr2=0x7fb4700ca380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:51.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.881+0000 7fb477f72640 1 --2- 172.21.15.131:0/3305551707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb47015bb80 0x7fb4700ca380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb470066570 tx=0x7fb46801cca0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.881+0000 7fb477f72640 1 -- 172.21.15.131:0/3305551707 shutdown_connections 2023-09-12T14:54:51.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.881+0000 7fb477f72640 1 --2- 172.21.15.131:0/3305551707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb47015bb80 0x7fb4700ca380 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.881+0000 7fb477f72640 1 --2- 172.21.15.131:0/3305551707 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4700b20b0 0x7fb47015b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.881+0000 7fb477f72640 1 --2- 172.21.15.131:0/3305551707 >> v2:172.21.15.187:3300/0 conn(0x7fb470157a40 0x7fb4700b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.881+0000 7fb477f72640 1 -- 172.21.15.131:0/3305551707 >> 172.21.15.131:0/3305551707 conn(0x7fb4700cd0d0 msgr2=0x7fb4700cd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:51.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.881+0000 7fb477f72640 1 -- 172.21.15.131:0/3305551707 shutdown_connections 2023-09-12T14:54:51.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.881+0000 7fb477f72640 1 -- 172.21.15.131:0/3305551707 wait complete. 2023-09-12T14:54:51.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.881+0000 7fb477f72640 1 Processor -- start 2023-09-12T14:54:51.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.881+0000 7fb477f72640 1 -- start start 2023-09-12T14:54:51.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb477f72640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4700b20b0 0x7fb47014d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:51.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb477f72640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb470157a40 0x7fb4700cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:51.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb477f72640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb47015bb80 0x7fb47013d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:51.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb477f72640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47013cea0 con 0x7fb4700b20b0 2023-09-12T14:54:51.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb477f72640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47013c870 con 0x7fb470157a40 2023-09-12T14:54:51.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb477f72640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb47014c7a0 con 0x7fb47015bb80 2023-09-12T14:54:51.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb475d0e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4700b20b0 0x7fb47014d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:51.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb475d0e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4700b20b0 0x7fb47014d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56120/0 (socket says 172.21.15.131:56120) 2023-09-12T14:54:51.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb475d0e640 1 -- 172.21.15.131:0/94540506 learned_addr learned my addr 172.21.15.131:0/94540506 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:51.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb47550d640 1 --2- 172.21.15.131:0/94540506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb470157a40 0x7fb4700cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:51.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb47650f640 1 --2- 172.21.15.131:0/94540506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb47015bb80 0x7fb47013d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:51.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb475d0e640 1 -- 172.21.15.131:0/94540506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb47015bb80 msgr2=0x7fb47013d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:51.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb475d0e640 1 --2- 172.21.15.131:0/94540506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb47015bb80 0x7fb47013d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb475d0e640 1 -- 172.21.15.131:0/94540506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb470157a40 msgr2=0x7fb4700cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:51.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb475d0e640 1 --2- 172.21.15.131:0/94540506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb470157a40 0x7fb4700cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:51.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb475d0e640 1 -- 172.21.15.131:0/94540506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4680098e0 con 0x7fb4700b20b0 2023-09-12T14:54:51.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb475d0e640 1 --2- 172.21.15.131:0/94540506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4700b20b0 0x7fb47014d9b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb46c00c970 tx=0x7fb46c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:51.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb4667fc640 1 -- 172.21.15.131:0/94540506 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c007bb0 con 0x7fb4700b20b0 2023-09-12T14:54:51.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb4667fc640 1 -- 172.21.15.131:0/94540506 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb46c007e00 con 0x7fb4700b20b0 2023-09-12T14:54:51.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb4667fc640 1 -- 172.21.15.131:0/94540506 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c005280 con 0x7fb4700b20b0 2023-09-12T14:54:51.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb477f72640 1 -- 172.21.15.131:0/94540506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb47014ca80 con 0x7fb4700b20b0 2023-09-12T14:54:51.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb477f72640 1 -- 172.21.15.131:0/94540506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb47013df20 con 0x7fb4700b20b0 2023-09-12T14:54:51.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb477f72640 1 -- 172.21.15.131:0/94540506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb470073580 con 0x7fb4700b20b0 2023-09-12T14:54:51.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb4667fc640 1 -- 172.21.15.131:0/94540506 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fb46c0053e0 con 0x7fb4700b20b0 2023-09-12T14:54:51.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb4667fc640 1 --2- 172.21.15.131:0/94540506 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb454065b50 0x7fb454068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:51.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb47550d640 1 --2- 172.21.15.131:0/94540506 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb454065b50 0x7fb454068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:51.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb4667fc640 1 -- 172.21.15.131:0/94540506 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb46c0835e0 con 0x7fb4700b20b0 2023-09-12T14:54:51.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.885+0000 7fb47550d640 1 --2- 172.21.15.131:0/94540506 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb454065b50 0x7fb454068010 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb470147ba0 tx=0x7fb46000a4e0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:51.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:51.889+0000 7fb4667fc640 1 -- 172.21.15.131:0/94540506 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fb46c054d10 con 0x7fb4700b20b0 2023-09-12T14:54:52.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:52.073+0000 7fb477f72640 1 -- 172.21.15.131:0/94540506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb47006a880 con 0x7fb4700b20b0 2023-09-12T14:54:52.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:52.077+0000 7fb4667fc640 1 -- 172.21.15.131:0/94540506 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1321 (secure 0 0 0) 0x7fb46c01d020 con 0x7fb4700b20b0 2023-09-12T14:54:52.078 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:54:52.079 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/91478708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":91478708},{"type":"v1","addr":"172.21.15.131:6836","nonce":91478708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-09-12T14:54:52.079 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 2 2023-09-12T14:54:52.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:52.077+0000 7fb477f72640 1 -- 172.21.15.131:0/94540506 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb454065b50 msgr2=0x7fb454068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:52.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:52.077+0000 7fb477f72640 1 --2- 172.21.15.131:0/94540506 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb454065b50 0x7fb454068010 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb470147ba0 tx=0x7fb46000a4e0 comp rx=0 tx=0).stop 2023-09-12T14:54:52.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:52.077+0000 7fb477f72640 1 -- 172.21.15.131:0/94540506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4700b20b0 msgr2=0x7fb47014d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:52.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:52.077+0000 7fb477f72640 1 --2- 172.21.15.131:0/94540506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4700b20b0 0x7fb47014d9b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb46c00c970 tx=0x7fb46c00ce30 comp rx=0 tx=0).stop 2023-09-12T14:54:52.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:52.077+0000 7fb477f72640 1 -- 172.21.15.131:0/94540506 shutdown_connections 2023-09-12T14:54:52.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:52.077+0000 7fb477f72640 1 --2- 172.21.15.131:0/94540506 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb454065b50 0x7fb454068010 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:52.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:52.077+0000 7fb477f72640 1 --2- 172.21.15.131:0/94540506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb47015bb80 0x7fb47013d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:52.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:52.077+0000 7fb477f72640 1 --2- 172.21.15.131:0/94540506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb470157a40 0x7fb4700cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:52.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:52.077+0000 7fb477f72640 1 --2- 172.21.15.131:0/94540506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4700b20b0 0x7fb47014d9b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:52.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:52.077+0000 7fb477f72640 1 -- 172.21.15.131:0/94540506 >> 172.21.15.131:0/94540506 conn(0x7fb4700cd0d0 msgr2=0x7fb4700ad710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:52.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:52.077+0000 7fb477f72640 1 -- 172.21.15.131:0/94540506 shutdown_connections 2023-09-12T14:54:52.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:52.077+0000 7fb477f72640 1 -- 172.21.15.131:0/94540506 wait complete. 2023-09-12T14:54:54.096 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T14:54:54.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.233+0000 7f46d1d18640 1 -- 172.21.15.131:0/3715819486 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46cc0aeb90 msgr2=0x7f46cc0af020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:54.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.233+0000 7f46d1d18640 1 --2- 172.21.15.131:0/3715819486 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46cc0aeb90 0x7f46cc0af020 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f46c4009c80 tx=0x7f46c401ccc0 comp rx=0 tx=0).stop 2023-09-12T14:54:54.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.233+0000 7f46d1d18640 1 -- 172.21.15.131:0/3715819486 shutdown_connections 2023-09-12T14:54:54.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.233+0000 7f46d1d18640 1 --2- 172.21.15.131:0/3715819486 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46cc0aeb90 0x7f46cc0af020 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:54.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.233+0000 7f46d1d18640 1 --2- 172.21.15.131:0/3715819486 >> v2:172.21.15.187:3300/0 conn(0x7f46cc0cd0c0 0x7f46cc0cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:54.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.233+0000 7f46d1d18640 1 -- 172.21.15.131:0/3715819486 >> 172.21.15.131:0/3715819486 conn(0x7f46cc0ceae0 msgr2=0x7f46cc0ceef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:54.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.233+0000 7f46d1d18640 1 -- 172.21.15.131:0/3715819486 shutdown_connections 2023-09-12T14:54:54.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.233+0000 7f46d1d18640 1 -- 172.21.15.131:0/3715819486 wait complete. 2023-09-12T14:54:54.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.233+0000 7f46d1d18640 1 Processor -- start 2023-09-12T14:54:54.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.233+0000 7f46d1d18640 1 -- start start 2023-09-12T14:54:54.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.237+0000 7f46d1d18640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46cc0aeb90 0x7f46cc14bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:54.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.237+0000 7f46d1d18640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46cc0cd0c0 0x7f46cc150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:54.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.237+0000 7f46d1d18640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46cc0cd4a0 0x7f46cc159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:54.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.237+0000 7f46d1d18640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc158850 con 0x7f46cc0cd0c0 2023-09-12T14:54:54.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.237+0000 7f46d1d18640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc14c4f0 con 0x7f46cc0aeb90 2023-09-12T14:54:54.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.237+0000 7f46d1d18640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc1577b0 con 0x7f46cc0cd4a0 2023-09-12T14:54:54.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.237+0000 7f46caffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46cc0cd0c0 0x7f46cc150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:54.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.237+0000 7f46caffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46cc0cd0c0 0x7f46cc150eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56132/0 (socket says 172.21.15.131:56132) 2023-09-12T14:54:54.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.237+0000 7f46caffd640 1 -- 172.21.15.131:0/1344226441 learned_addr learned my addr 172.21.15.131:0/1344226441 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:54.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.237+0000 7f46cbfff640 1 --2- 172.21.15.131:0/1344226441 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46cc0cd4a0 0x7f46cc159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:54.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.237+0000 7f46cb7fe640 1 --2- 172.21.15.131:0/1344226441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46cc0aeb90 0x7f46cc14bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:54.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.237+0000 7f46caffd640 1 -- 172.21.15.131:0/1344226441 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46cc0cd4a0 msgr2=0x7f46cc159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:54.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.237+0000 7f46caffd640 1 --2- 172.21.15.131:0/1344226441 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46cc0cd4a0 0x7f46cc159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:54.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.237+0000 7f46caffd640 1 -- 172.21.15.131:0/1344226441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46cc0aeb90 msgr2=0x7f46cc14bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:54.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.237+0000 7f46caffd640 1 --2- 172.21.15.131:0/1344226441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46cc0aeb90 0x7f46cc14bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:54.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.237+0000 7f46caffd640 1 -- 172.21.15.131:0/1344226441 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c40098e0 con 0x7f46cc0cd0c0 2023-09-12T14:54:54.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.237+0000 7f46caffd640 1 --2- 172.21.15.131:0/1344226441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46cc0cd0c0 0x7f46cc150eb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f46c000c370 tx=0x7f46c000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:54.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.237+0000 7f46b3fff640 1 -- 172.21.15.131:0/1344226441 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c00052d0 con 0x7f46cc0cd0c0 2023-09-12T14:54:54.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.237+0000 7f46d1d18640 1 -- 172.21.15.131:0/1344226441 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46cc157a90 con 0x7f46cc0cd0c0 2023-09-12T14:54:54.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.237+0000 7f46b3fff640 1 -- 172.21.15.131:0/1344226441 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f46c0007020 con 0x7f46cc0cd0c0 2023-09-12T14:54:54.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.237+0000 7f46b3fff640 1 -- 172.21.15.131:0/1344226441 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c0007430 con 0x7f46cc0cd0c0 2023-09-12T14:54:54.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.237+0000 7f46d1d18640 1 -- 172.21.15.131:0/1344226441 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46cc158050 con 0x7f46cc0cd0c0 2023-09-12T14:54:54.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.237+0000 7f46d1d18640 1 -- 172.21.15.131:0/1344226441 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46cc073580 con 0x7f46cc0cd0c0 2023-09-12T14:54:54.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.237+0000 7f46b3fff640 1 -- 172.21.15.131:0/1344226441 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f46c0007590 con 0x7f46cc0cd0c0 2023-09-12T14:54:54.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.237+0000 7f46b3fff640 1 --2- 172.21.15.131:0/1344226441 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f46b8065b50 0x7f46b8068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:54.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.237+0000 7f46b3fff640 1 -- 172.21.15.131:0/1344226441 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f46c0082580 con 0x7f46cc0cd0c0 2023-09-12T14:54:54.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.237+0000 7f46cb7fe640 1 --2- 172.21.15.131:0/1344226441 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f46b8065b50 0x7f46b8068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:54.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.241+0000 7f46cb7fe640 1 --2- 172.21.15.131:0/1344226441 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f46b8065b50 0x7f46b8068010 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009be0 tx=0x7f46bc0091e0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:54.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.241+0000 7f46b3fff640 1 -- 172.21.15.131:0/1344226441 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f46c0050250 con 0x7f46cc0cd0c0 2023-09-12T14:54:54.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.413+0000 7f46d1d18640 1 -- 172.21.15.131:0/1344226441 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46cc0cd8e0 con 0x7f46cc0cd0c0 2023-09-12T14:54:54.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.417+0000 7f46b3fff640 1 -- 172.21.15.131:0/1344226441 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3694 (secure 0 0 0) 0x7f46c0053870 con 0x7f46cc0cd0c0 2023-09-12T14:54:54.418 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:54:54.418 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4344,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/491327908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":491327908},{"type":"v1","addr":"172.21.15.131:6837","nonce":491327908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3470212105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3470212105},{"type":"v1","addr":"172.21.15.187:6833","nonce":3470212105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/91478708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":91478708},{"type":"v1","addr":"172.21.15.131:6836","nonce":91478708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4396,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/290727181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":290727181},{"type":"v1","addr":"172.21.15.187:6835","nonce":290727181}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-09-12T14:54:54.419 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 3 2023-09-12T14:54:54.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.417+0000 7f46d1d18640 1 -- 172.21.15.131:0/1344226441 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f46b8065b50 msgr2=0x7f46b8068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:54.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.417+0000 7f46d1d18640 1 --2- 172.21.15.131:0/1344226441 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f46b8065b50 0x7f46b8068010 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009be0 tx=0x7f46bc0091e0 comp rx=0 tx=0).stop 2023-09-12T14:54:54.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.417+0000 7f46d1d18640 1 -- 172.21.15.131:0/1344226441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46cc0cd0c0 msgr2=0x7f46cc150eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:54.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.417+0000 7f46d1d18640 1 --2- 172.21.15.131:0/1344226441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46cc0cd0c0 0x7f46cc150eb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f46c000c370 tx=0x7f46c000c830 comp rx=0 tx=0).stop 2023-09-12T14:54:54.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.417+0000 7f46d1d18640 1 -- 172.21.15.131:0/1344226441 shutdown_connections 2023-09-12T14:54:54.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.417+0000 7f46d1d18640 1 --2- 172.21.15.131:0/1344226441 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f46b8065b50 0x7f46b8068010 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:54.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.417+0000 7f46d1d18640 1 --2- 172.21.15.131:0/1344226441 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46cc0cd4a0 0x7f46cc159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:54.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.417+0000 7f46d1d18640 1 --2- 172.21.15.131:0/1344226441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46cc0cd0c0 0x7f46cc150eb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:54.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.417+0000 7f46d1d18640 1 --2- 172.21.15.131:0/1344226441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46cc0aeb90 0x7f46cc14bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:54.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.417+0000 7f46d1d18640 1 -- 172.21.15.131:0/1344226441 >> 172.21.15.131:0/1344226441 conn(0x7f46cc0ceae0 msgr2=0x7f46cc0ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:54.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.417+0000 7f46d1d18640 1 -- 172.21.15.131:0/1344226441 shutdown_connections 2023-09-12T14:54:54.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.417+0000 7f46d1d18640 1 -- 172.21.15.131:0/1344226441 wait complete. 2023-09-12T14:54:54.435 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:54:54.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7aa83f640 1 -- 172.21.15.131:0/4104656581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7a40afa40 msgr2=0x7fa7a40afed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:54.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7aa83f640 1 --2- 172.21.15.131:0/4104656581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7a40afa40 0x7fa7a40afed0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa7a406fe90 tx=0x7fa79c01ccd0 comp rx=0 tx=0).stop 2023-09-12T14:54:54.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7aa83f640 1 -- 172.21.15.131:0/4104656581 shutdown_connections 2023-09-12T14:54:54.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7aa83f640 1 --2- 172.21.15.131:0/4104656581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7a40afa40 0x7fa7a40afed0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:54.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7aa83f640 1 --2- 172.21.15.131:0/4104656581 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7a406db90 0x7fa7a415c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:54.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7aa83f640 1 --2- 172.21.15.131:0/4104656581 >> v2:172.21.15.187:3300/0 conn(0x7fa7a4157bc0 0x7fa7a4070a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:54.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7aa83f640 1 -- 172.21.15.131:0/4104656581 >> 172.21.15.131:0/4104656581 conn(0x7fa7a40cdd80 msgr2=0x7fa7a40ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:54.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7aa83f640 1 -- 172.21.15.131:0/4104656581 shutdown_connections 2023-09-12T14:54:54.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7aa83f640 1 -- 172.21.15.131:0/4104656581 wait complete. 2023-09-12T14:54:54.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7aa83f640 1 Processor -- start 2023-09-12T14:54:54.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7aa83f640 1 -- start start 2023-09-12T14:54:54.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7aa83f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7a406db90 0x7fa7a4156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:54.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7aa83f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7a40afa40 0x7fa7a41f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:54.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7aa83f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7a4157bc0 0x7fa7a41498c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:54.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7aa83f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a4156560 con 0x7fa7a4157bc0 2023-09-12T14:54:54.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7aa83f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a4157190 con 0x7fa7a40afa40 2023-09-12T14:54:54.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7aa83f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a41f5570 con 0x7fa7a406db90 2023-09-12T14:54:54.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7a8ddc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7a4157bc0 0x7fa7a41498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:54.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7a8ddc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7a4157bc0 0x7fa7a41498c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56158/0 (socket says 172.21.15.131:56158) 2023-09-12T14:54:54.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7a8ddc640 1 -- 172.21.15.131:0/2832970384 learned_addr learned my addr 172.21.15.131:0/2832970384 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:54.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7a37fe640 1 --2- 172.21.15.131:0/2832970384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7a40afa40 0x7fa7a41f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:54.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7a8ddc640 1 -- 172.21.15.131:0/2832970384 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7a406db90 msgr2=0x7fa7a4156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:54.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7a8ddc640 1 --2- 172.21.15.131:0/2832970384 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7a406db90 0x7fa7a4156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:54.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7a8ddc640 1 -- 172.21.15.131:0/2832970384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7a40afa40 msgr2=0x7fa7a41f5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:54.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7a8ddc640 1 --2- 172.21.15.131:0/2832970384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7a40afa40 0x7fa7a41f5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:54.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7a8ddc640 1 -- 172.21.15.131:0/2832970384 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79c0098e0 con 0x7fa7a4157bc0 2023-09-12T14:54:54.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7a8ddc640 1 --2- 172.21.15.131:0/2832970384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7a4157bc0 0x7fa7a41498c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa79c009bd0 tx=0x7fa79c0a5ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:54.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7a0ff9640 1 -- 172.21.15.131:0/2832970384 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c004e60 con 0x7fa7a4157bc0 2023-09-12T14:54:54.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7a0ff9640 1 -- 172.21.15.131:0/2832970384 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa79c004050 con 0x7fa7a4157bc0 2023-09-12T14:54:54.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7aa83f640 1 -- 172.21.15.131:0/2832970384 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a4155b90 con 0x7fa7a4157bc0 2023-09-12T14:54:54.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7a0ff9640 1 -- 172.21.15.131:0/2832970384 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c004420 con 0x7fa7a4157bc0 2023-09-12T14:54:54.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7aa83f640 1 -- 172.21.15.131:0/2832970384 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a4155540 con 0x7fa7a4157bc0 2023-09-12T14:54:54.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.557+0000 7fa7aa83f640 1 -- 172.21.15.131:0/2832970384 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a40ca600 con 0x7fa7a4157bc0 2023-09-12T14:54:54.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.561+0000 7fa7a0ff9640 1 -- 172.21.15.131:0/2832970384 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fa79c0b4040 con 0x7fa7a4157bc0 2023-09-12T14:54:54.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.561+0000 7fa7a0ff9640 1 --2- 172.21.15.131:0/2832970384 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa790065b20 0x7fa790067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:54.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.561+0000 7fa7a0ff9640 1 -- 172.21.15.131:0/2832970384 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa79c11d460 con 0x7fa7a4157bc0 2023-09-12T14:54:54.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.561+0000 7fa7a3fff640 1 --2- 172.21.15.131:0/2832970384 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa790065b20 0x7fa790067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:54.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.561+0000 7fa7a3fff640 1 --2- 172.21.15.131:0/2832970384 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa790065b20 0x7fa790067fe0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa79400a860 tx=0x7fa794005b60 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:54.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.565+0000 7fa7a0ff9640 1 -- 172.21.15.131:0/2832970384 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fa79c0eb190 con 0x7fa7a4157bc0 2023-09-12T14:54:54.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.709+0000 7fa7aa83f640 1 -- 172.21.15.131:0/2832970384 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7a406c800 con 0x7fa7a4157bc0 2023-09-12T14:54:54.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.713+0000 7fa7a0ff9640 1 -- 172.21.15.131:0/2832970384 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12886 (secure 0 0 0) 0x7fa79c0b2090 con 0x7fa7a4157bc0 2023-09-12T14:54:54.714 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:54:54.714 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":16,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:54:47.872696+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:54:54.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.713+0000 7fa7aa83f640 1 -- 172.21.15.131:0/2832970384 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa790065b20 msgr2=0x7fa790067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:54.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.713+0000 7fa7aa83f640 1 --2- 172.21.15.131:0/2832970384 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa790065b20 0x7fa790067fe0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa79400a860 tx=0x7fa794005b60 comp rx=0 tx=0).stop 2023-09-12T14:54:54.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.713+0000 7fa7aa83f640 1 -- 172.21.15.131:0/2832970384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7a4157bc0 msgr2=0x7fa7a41498c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:54.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.713+0000 7fa7aa83f640 1 --2- 172.21.15.131:0/2832970384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7a4157bc0 0x7fa7a41498c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa79c009bd0 tx=0x7fa79c0a5ac0 comp rx=0 tx=0).stop 2023-09-12T14:54:54.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.713+0000 7fa7aa83f640 1 -- 172.21.15.131:0/2832970384 shutdown_connections 2023-09-12T14:54:54.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.713+0000 7fa7aa83f640 1 --2- 172.21.15.131:0/2832970384 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa790065b20 0x7fa790067fe0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:54.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.713+0000 7fa7aa83f640 1 --2- 172.21.15.131:0/2832970384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7a4157bc0 0x7fa7a41498c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:54.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.713+0000 7fa7aa83f640 1 --2- 172.21.15.131:0/2832970384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7a40afa40 0x7fa7a41f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:54.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.713+0000 7fa7aa83f640 1 --2- 172.21.15.131:0/2832970384 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7a406db90 0x7fa7a4156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:54.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.713+0000 7fa7aa83f640 1 -- 172.21.15.131:0/2832970384 >> 172.21.15.131:0/2832970384 conn(0x7fa7a40cdd80 msgr2=0x7fa7a406cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:54.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.713+0000 7fa7aa83f640 1 -- 172.21.15.131:0/2832970384 shutdown_connections 2023-09-12T14:54:54.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.713+0000 7fa7aa83f640 1 -- 172.21.15.131:0/2832970384 wait complete. 2023-09-12T14:54:54.731 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T14:54:45.654004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-12T14:54:54.732 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T14:54:54.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.849+0000 7f9512591640 1 -- 172.21.15.131:0/1721846194 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f950c06b050 msgr2=0x7f950c06b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:54.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.849+0000 7f9512591640 1 --2- 172.21.15.131:0/1721846194 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f950c06b050 0x7f950c06b4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9504009bc0 tx=0x7f950401cb50 comp rx=0 tx=0).stop 2023-09-12T14:54:54.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.849+0000 7f9512591640 1 -- 172.21.15.131:0/1721846194 shutdown_connections 2023-09-12T14:54:54.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.849+0000 7f9512591640 1 --2- 172.21.15.131:0/1721846194 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f950c06b050 0x7f950c06b4a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:54.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.849+0000 7f9512591640 1 --2- 172.21.15.131:0/1721846194 >> v2:172.21.15.187:3300/0 conn(0x7f950c0ab730 0x7f950c0abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:54.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.849+0000 7f9512591640 1 -- 172.21.15.131:0/1721846194 >> 172.21.15.131:0/1721846194 conn(0x7f950c069840 msgr2=0x7f950c0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:54.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.849+0000 7f9512591640 1 -- 172.21.15.131:0/1721846194 shutdown_connections 2023-09-12T14:54:54.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.849+0000 7f9512591640 1 -- 172.21.15.131:0/1721846194 wait complete. 2023-09-12T14:54:54.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.849+0000 7f9512591640 1 Processor -- start 2023-09-12T14:54:54.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.849+0000 7f9512591640 1 -- start start 2023-09-12T14:54:54.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.849+0000 7f9512591640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f950c06b050 0x7f950c154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:54.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.849+0000 7f9512591640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f950c0ab730 0x7f950c0ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:54.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.849+0000 7f9512591640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f950c158180 0x7f950c0ad350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:54.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.849+0000 7f9512591640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950c154e50 con 0x7f950c0ab730 2023-09-12T14:54:54.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.849+0000 7f9512591640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950c1570f0 con 0x7f950c06b050 2023-09-12T14:54:54.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.849+0000 7f9512591640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f950c156240 con 0x7f950c158180 2023-09-12T14:54:54.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.849+0000 7f9503fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f950c0ab730 0x7f950c0ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:54.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.849+0000 7f9503fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f950c0ab730 0x7f950c0ace10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56168/0 (socket says 172.21.15.131:56168) 2023-09-12T14:54:54.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.849+0000 7f9503fff640 1 -- 172.21.15.131:0/1308219507 learned_addr learned my addr 172.21.15.131:0/1308219507 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:54.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.849+0000 7f950bfff640 1 --2- 172.21.15.131:0/1308219507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f950c06b050 0x7f950c154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:54.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.853+0000 7f9510b2e640 1 --2- 172.21.15.131:0/1308219507 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f950c158180 0x7f950c0ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:54.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.853+0000 7f9503fff640 1 -- 172.21.15.131:0/1308219507 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f950c158180 msgr2=0x7f950c0ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:54.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.853+0000 7f9503fff640 1 --2- 172.21.15.131:0/1308219507 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f950c158180 0x7f950c0ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:54.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.853+0000 7f9503fff640 1 -- 172.21.15.131:0/1308219507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f950c06b050 msgr2=0x7f950c154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:54.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.853+0000 7f9503fff640 1 --2- 172.21.15.131:0/1308219507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f950c06b050 0x7f950c154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:54.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.853+0000 7f9503fff640 1 -- 172.21.15.131:0/1308219507 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9504009820 con 0x7f950c0ab730 2023-09-12T14:54:54.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.853+0000 7f950bfff640 1 --2- 172.21.15.131:0/1308219507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f950c06b050 0x7f950c154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:54:54.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.853+0000 7f9503fff640 1 --2- 172.21.15.131:0/1308219507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f950c0ab730 0x7f950c0ace10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f94f400c340 tx=0x7f94f400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:54.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.853+0000 7f95097fa640 1 -- 172.21.15.131:0/1308219507 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f4004230 con 0x7f950c0ab730 2023-09-12T14:54:54.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.853+0000 7f95097fa640 1 -- 172.21.15.131:0/1308219507 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f94f4004480 con 0x7f950c0ab730 2023-09-12T14:54:54.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.853+0000 7f9512591640 1 -- 172.21.15.131:0/1308219507 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f950c1564e0 con 0x7f950c0ab730 2023-09-12T14:54:54.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.853+0000 7f95097fa640 1 -- 172.21.15.131:0/1308219507 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f4003860 con 0x7f950c0ab730 2023-09-12T14:54:54.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.853+0000 7f9512591640 1 -- 172.21.15.131:0/1308219507 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f950c156aa0 con 0x7f950c0ab730 2023-09-12T14:54:54.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.853+0000 7f9512591640 1 -- 172.21.15.131:0/1308219507 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f950c1572a0 con 0x7f950c0ab730 2023-09-12T14:54:54.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.853+0000 7f95097fa640 1 -- 172.21.15.131:0/1308219507 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f94f40039c0 con 0x7f950c0ab730 2023-09-12T14:54:54.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.853+0000 7f95097fa640 1 --2- 172.21.15.131:0/1308219507 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f94ec065b20 0x7f94ec067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:54.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.853+0000 7f95097fa640 1 -- 172.21.15.131:0/1308219507 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f94f4082750 con 0x7f950c0ab730 2023-09-12T14:54:54.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.853+0000 7f950bfff640 1 --2- 172.21.15.131:0/1308219507 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f94ec065b20 0x7f94ec067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:54.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.853+0000 7f950bfff640 1 --2- 172.21.15.131:0/1308219507 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f94ec065b20 0x7f94ec067fe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009c30 tx=0x7f94fc0091e0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:54.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:54.857+0000 7f95097fa640 1 -- 172.21.15.131:0/1308219507 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f950c1572a0 con 0x7f950c0ab730 2023-09-12T14:54:55.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.013+0000 7f9512591640 1 -- 172.21.15.131:0/1308219507 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f950c157260 con 0x7f950c0ab730 2023-09-12T14:54:55.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.013+0000 7f95097fa640 1 -- 172.21.15.131:0/1308219507 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f94f4050370 con 0x7f950c0ab730 2023-09-12T14:54:55.014 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-09-12T14:54:55.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.013+0000 7f9512591640 1 -- 172.21.15.131:0/1308219507 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f94ec065b20 msgr2=0x7f94ec067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:55.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.013+0000 7f9512591640 1 --2- 172.21.15.131:0/1308219507 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f94ec065b20 0x7f94ec067fe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009c30 tx=0x7f94fc0091e0 comp rx=0 tx=0).stop 2023-09-12T14:54:55.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.013+0000 7f9512591640 1 -- 172.21.15.131:0/1308219507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f950c0ab730 msgr2=0x7f950c0ace10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:55.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.013+0000 7f9512591640 1 --2- 172.21.15.131:0/1308219507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f950c0ab730 0x7f950c0ace10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f94f400c340 tx=0x7f94f400c800 comp rx=0 tx=0).stop 2023-09-12T14:54:55.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.013+0000 7f9512591640 1 -- 172.21.15.131:0/1308219507 shutdown_connections 2023-09-12T14:54:55.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.013+0000 7f9512591640 1 --2- 172.21.15.131:0/1308219507 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f94ec065b20 0x7f94ec067fe0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:55.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.013+0000 7f9512591640 1 --2- 172.21.15.131:0/1308219507 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f950c158180 0x7f950c0ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:55.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.013+0000 7f9512591640 1 --2- 172.21.15.131:0/1308219507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f950c0ab730 0x7f950c0ace10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:55.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.013+0000 7f9512591640 1 --2- 172.21.15.131:0/1308219507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f950c06b050 0x7f950c154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:55.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.013+0000 7f9512591640 1 -- 172.21.15.131:0/1308219507 >> 172.21.15.131:0/1308219507 conn(0x7f950c069840 msgr2=0x7f950c0adf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:55.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.013+0000 7f9512591640 1 -- 172.21.15.131:0/1308219507 shutdown_connections 2023-09-12T14:54:55.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.013+0000 7f9512591640 1 -- 172.21.15.131:0/1308219507 wait complete. 2023-09-12T14:54:55.033 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-12T14:54:55.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.145+0000 7f0bffde3640 1 -- 172.21.15.131:0/1899299527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bf806b050 msgr2=0x7f0bf806b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:55.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.145+0000 7f0bffde3640 1 --2- 172.21.15.131:0/1899299527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bf806b050 0x7f0bf806b4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0009bc0 tx=0x7f0bf001cb50 comp rx=0 tx=0).stop 2023-09-12T14:54:55.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.145+0000 7f0bffde3640 1 -- 172.21.15.131:0/1899299527 shutdown_connections 2023-09-12T14:54:55.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.145+0000 7f0bffde3640 1 --2- 172.21.15.131:0/1899299527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bf806b050 0x7f0bf806b4a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:55.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.145+0000 7f0bffde3640 1 --2- 172.21.15.131:0/1899299527 >> v2:172.21.15.187:3300/0 conn(0x7f0bf80ab730 0x7f0bf80abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:55.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.145+0000 7f0bffde3640 1 -- 172.21.15.131:0/1899299527 >> 172.21.15.131:0/1899299527 conn(0x7f0bf8069840 msgr2=0x7f0bf80cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:55.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.145+0000 7f0bffde3640 1 -- 172.21.15.131:0/1899299527 shutdown_connections 2023-09-12T14:54:55.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.145+0000 7f0bffde3640 1 -- 172.21.15.131:0/1899299527 wait complete. 2023-09-12T14:54:55.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.145+0000 7f0bffde3640 1 Processor -- start 2023-09-12T14:54:55.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.145+0000 7f0bffde3640 1 -- start start 2023-09-12T14:54:55.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.145+0000 7f0bffde3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bf806b050 0x7f0bf8132960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:55.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.149+0000 7f0bffde3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bf80ab730 0x7f0bf8159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:55.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.149+0000 7f0bffde3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bf8158180 0x7f0bf8133270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:55.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.149+0000 7f0bffde3640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf814a0d0 con 0x7f0bf806b050 2023-09-12T14:54:55.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.149+0000 7f0bffde3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf81456b0 con 0x7f0bf80ab730 2023-09-12T14:54:55.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.149+0000 7f0bffde3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf8159240 con 0x7f0bf8158180 2023-09-12T14:54:55.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.149+0000 7f0bfdb7f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bf806b050 0x7f0bf8132960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:55.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.149+0000 7f0bfdb7f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bf806b050 0x7f0bf8132960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56174/0 (socket says 172.21.15.131:56174) 2023-09-12T14:54:55.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.149+0000 7f0bfdb7f640 1 -- 172.21.15.131:0/3291015615 learned_addr learned my addr 172.21.15.131:0/3291015615 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:55.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.149+0000 7f0bfd37e640 1 --2- 172.21.15.131:0/3291015615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bf80ab730 0x7f0bf8159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:55.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.149+0000 7f0bfdb7f640 1 -- 172.21.15.131:0/3291015615 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bf8158180 msgr2=0x7f0bf8133270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:55.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.149+0000 7f0bfdb7f640 1 --2- 172.21.15.131:0/3291015615 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bf8158180 0x7f0bf8133270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:55.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.149+0000 7f0bfdb7f640 1 -- 172.21.15.131:0/3291015615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bf80ab730 msgr2=0x7f0bf8159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:55.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.149+0000 7f0bfdb7f640 1 --2- 172.21.15.131:0/3291015615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bf80ab730 0x7f0bf8159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:55.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.149+0000 7f0bfdb7f640 1 -- 172.21.15.131:0/3291015615 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf0009820 con 0x7f0bf806b050 2023-09-12T14:54:55.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.149+0000 7f0bfd37e640 1 --2- 172.21.15.131:0/3291015615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bf80ab730 0x7f0bf8159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:54:55.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.149+0000 7f0bfdb7f640 1 --2- 172.21.15.131:0/3291015615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bf806b050 0x7f0bf8132960 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400e9f0 tx=0x7f0bf400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:55.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.149+0000 7f0bee7fc640 1 -- 172.21.15.131:0/3291015615 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4009710 con 0x7f0bf806b050 2023-09-12T14:54:55.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.149+0000 7f0bee7fc640 1 -- 172.21.15.131:0/3291015615 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0bf400ce10 con 0x7f0bf806b050 2023-09-12T14:54:55.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.149+0000 7f0bffde3640 1 -- 172.21.15.131:0/3291015615 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf8159800 con 0x7f0bf806b050 2023-09-12T14:54:55.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.149+0000 7f0bee7fc640 1 -- 172.21.15.131:0/3291015615 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4010690 con 0x7f0bf806b050 2023-09-12T14:54:55.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.149+0000 7f0bffde3640 1 -- 172.21.15.131:0/3291015615 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf8159490 con 0x7f0bf806b050 2023-09-12T14:54:55.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.149+0000 7f0bffde3640 1 -- 172.21.15.131:0/3291015615 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf8073580 con 0x7f0bf806b050 2023-09-12T14:54:55.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.149+0000 7f0bee7fc640 1 -- 172.21.15.131:0/3291015615 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f0bf40107f0 con 0x7f0bf806b050 2023-09-12T14:54:55.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.149+0000 7f0bee7fc640 1 --2- 172.21.15.131:0/3291015615 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0bdc065b20 0x7f0bdc067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:55.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.149+0000 7f0bee7fc640 1 -- 172.21.15.131:0/3291015615 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0bf4014070 con 0x7f0bf806b050 2023-09-12T14:54:55.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.149+0000 7f0bfd37e640 1 --2- 172.21.15.131:0/3291015615 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0bdc065b20 0x7f0bdc067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:55.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.149+0000 7f0bfd37e640 1 --2- 172.21.15.131:0/3291015615 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0bdc065b20 0x7f0bdc067fe0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80aede0 tx=0x7f0be8009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:55.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.153+0000 7f0bee7fc640 1 -- 172.21.15.131:0/3291015615 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f0bf404fdb0 con 0x7f0bf806b050 2023-09-12T14:54:55.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.305+0000 7f0bffde3640 1 -- 172.21.15.131:0/3291015615 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0bf80b2440 con 0x7f0bf806b050 2023-09-12T14:54:55.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.305+0000 7f0bee7fc640 1 -- 172.21.15.131:0/3291015615 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0bf4055350 con 0x7f0bf806b050 2023-09-12T14:54:55.307 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:54:55.308 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2023-09-12T14:54:55.308 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-09-12T14:54:55.308 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2023-09-12T14:54:55.308 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2023-09-12T14:54:55.309 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:54:55.309 INFO:teuthology.orchestra.run.smithi131.stdout:] 2023-09-12T14:54:55.309 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-12T14:54:55.310 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.305+0000 7f0bffde3640 1 -- 172.21.15.131:0/3291015615 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0bdc065b20 msgr2=0x7f0bdc067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:55.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.305+0000 7f0bffde3640 1 --2- 172.21.15.131:0/3291015615 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0bdc065b20 0x7f0bdc067fe0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80aede0 tx=0x7f0be8009040 comp rx=0 tx=0).stop 2023-09-12T14:54:55.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.305+0000 7f0bffde3640 1 -- 172.21.15.131:0/3291015615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bf806b050 msgr2=0x7f0bf8132960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:55.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.305+0000 7f0bffde3640 1 --2- 172.21.15.131:0/3291015615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bf806b050 0x7f0bf8132960 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400e9f0 tx=0x7f0bf400eeb0 comp rx=0 tx=0).stop 2023-09-12T14:54:55.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.305+0000 7f0bffde3640 1 -- 172.21.15.131:0/3291015615 shutdown_connections 2023-09-12T14:54:55.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.305+0000 7f0bffde3640 1 --2- 172.21.15.131:0/3291015615 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0bdc065b20 0x7f0bdc067fe0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:55.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.305+0000 7f0bffde3640 1 --2- 172.21.15.131:0/3291015615 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bf8158180 0x7f0bf8133270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:55.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.305+0000 7f0bffde3640 1 --2- 172.21.15.131:0/3291015615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bf80ab730 0x7f0bf8159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:55.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.305+0000 7f0bffde3640 1 --2- 172.21.15.131:0/3291015615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bf806b050 0x7f0bf8132960 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:55.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.305+0000 7f0bffde3640 1 -- 172.21.15.131:0/3291015615 >> 172.21.15.131:0/3291015615 conn(0x7f0bf8069840 msgr2=0x7f0bf80cac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:55.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.305+0000 7f0bffde3640 1 -- 172.21.15.131:0/3291015615 shutdown_connections 2023-09-12T14:54:55.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.305+0000 7f0bffde3640 1 -- 172.21.15.131:0/3291015615 wait complete. 2023-09-12T14:54:55.314 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-12T14:54:55.324 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-12T14:54:55.325 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-12T14:54:55.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ac2325640 1 -- 172.21.15.131:0/2052833365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6abc158180 msgr2=0x7f6abc0cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:55.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ac2325640 1 --2- 172.21.15.131:0/2052833365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6abc158180 0x7f6abc0cae70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6abc06eea0 tx=0x7f6aa401c830 comp rx=0 tx=0).stop 2023-09-12T14:54:55.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ac2325640 1 -- 172.21.15.131:0/2052833365 shutdown_connections 2023-09-12T14:54:55.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ac2325640 1 --2- 172.21.15.131:0/2052833365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6abc158180 0x7f6abc0cae70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:55.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ac2325640 1 --2- 172.21.15.131:0/2052833365 >> v2:172.21.15.187:3300/0 conn(0x7f6abc0ab730 0x7f6abc0abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:55.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ac2325640 1 -- 172.21.15.131:0/2052833365 >> 172.21.15.131:0/2052833365 conn(0x7f6abc069840 msgr2=0x7f6abc0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:55.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ac2325640 1 -- 172.21.15.131:0/2052833365 shutdown_connections 2023-09-12T14:54:55.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ac2325640 1 -- 172.21.15.131:0/2052833365 wait complete. 2023-09-12T14:54:55.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ac2325640 1 Processor -- start 2023-09-12T14:54:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ac2325640 1 -- start start 2023-09-12T14:54:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ac2325640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6abc06f390 0x7f6abc1478e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ac2325640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6abc0ab730 0x7f6abc146250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ac2325640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6abc158180 0x7f6abc14a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ac2325640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc147e20 con 0x7f6abc158180 2023-09-12T14:54:55.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ac2325640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc146790 con 0x7f6abc06f390 2023-09-12T14:54:55.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ac2325640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc15b1d0 con 0x7f6abc0ab730 2023-09-12T14:54:55.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ac08c2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6abc158180 0x7f6abc14a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:55.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ac08c2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6abc158180 0x7f6abc14a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56188/0 (socket says 172.21.15.131:56188) 2023-09-12T14:54:55.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ac08c2640 1 -- 172.21.15.131:0/3592831975 learned_addr learned my addr 172.21.15.131:0/3592831975 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:55.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6abb7fe640 1 --2- 172.21.15.131:0/3592831975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6abc0ab730 0x7f6abc146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:55.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ac08c2640 1 -- 172.21.15.131:0/3592831975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6abc0ab730 msgr2=0x7f6abc146250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:55.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6abbfff640 1 --2- 172.21.15.131:0/3592831975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6abc06f390 0x7f6abc1478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:55.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ac08c2640 1 --2- 172.21.15.131:0/3592831975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6abc0ab730 0x7f6abc146250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:55.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ac08c2640 1 -- 172.21.15.131:0/3592831975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6abc06f390 msgr2=0x7f6abc1478e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:55.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ac08c2640 1 --2- 172.21.15.131:0/3592831975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6abc06f390 0x7f6abc1478e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:55.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ac08c2640 1 -- 172.21.15.131:0/3592831975 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa4009560 con 0x7f6abc158180 2023-09-12T14:54:55.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6abb7fe640 1 --2- 172.21.15.131:0/3592831975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6abc0ab730 0x7f6abc146250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:54:55.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ac08c2640 1 --2- 172.21.15.131:0/3592831975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6abc158180 0x7f6abc14a0d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6ab400c780 tx=0x7f6ab400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:55.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ab8ff9640 1 -- 172.21.15.131:0/3592831975 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4003d90 con 0x7f6abc158180 2023-09-12T14:54:55.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ab8ff9640 1 -- 172.21.15.131:0/3592831975 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6ab400ce90 con 0x7f6abc158180 2023-09-12T14:54:55.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ac2325640 1 -- 172.21.15.131:0/3592831975 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6abc15a5c0 con 0x7f6abc158180 2023-09-12T14:54:55.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ab8ff9640 1 -- 172.21.15.131:0/3592831975 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab40052d0 con 0x7f6abc158180 2023-09-12T14:54:55.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.489+0000 7f6ac2325640 1 -- 172.21.15.131:0/3592831975 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6abc15ab80 con 0x7f6abc158180 2023-09-12T14:54:55.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.493+0000 7f6ac2325640 1 -- 172.21.15.131:0/3592831975 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6abc14b460 con 0x7f6abc158180 2023-09-12T14:54:55.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.493+0000 7f6ab8ff9640 1 -- 172.21.15.131:0/3592831975 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f6ab4005430 con 0x7f6abc158180 2023-09-12T14:54:55.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.493+0000 7f6ab8ff9640 1 --2- 172.21.15.131:0/3592831975 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6a9c065b20 0x7f6a9c067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:55.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.493+0000 7f6ab8ff9640 1 -- 172.21.15.131:0/3592831975 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6ab4082720 con 0x7f6abc158180 2023-09-12T14:54:55.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.493+0000 7f6abbfff640 1 --2- 172.21.15.131:0/3592831975 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6a9c065b20 0x7f6a9c067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:55.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.493+0000 7f6abbfff640 1 --2- 172.21.15.131:0/3592831975 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6a9c065b20 0x7f6a9c067fe0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6abc072490 tx=0x7f6aac008040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:55.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.497+0000 7f6ab8ff9640 1 -- 172.21.15.131:0/3592831975 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f6abc14b460 con 0x7f6abc158180 2023-09-12T14:54:55.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:55.649+0000 7f6ac2325640 1 -- 172.21.15.131:0/3592831975 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6abc14b460 con 0x7f6abc158180 2023-09-12T14:54:56.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.653+0000 7f6ab8ff9640 1 -- 172.21.15.131:0/3592831975 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6abc14b460 con 0x7f6abc158180 2023-09-12T14:54:56.656 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2023-09-12T14:54:56.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.657+0000 7f6ac2325640 1 -- 172.21.15.131:0/3592831975 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6a9c065b20 msgr2=0x7f6a9c067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:56.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.657+0000 7f6ac2325640 1 --2- 172.21.15.131:0/3592831975 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6a9c065b20 0x7f6a9c067fe0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6abc072490 tx=0x7f6aac008040 comp rx=0 tx=0).stop 2023-09-12T14:54:56.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.657+0000 7f6ac2325640 1 -- 172.21.15.131:0/3592831975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6abc158180 msgr2=0x7f6abc14a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:56.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.657+0000 7f6ac2325640 1 --2- 172.21.15.131:0/3592831975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6abc158180 0x7f6abc14a0d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6ab400c780 tx=0x7f6ab400cc40 comp rx=0 tx=0).stop 2023-09-12T14:54:56.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.657+0000 7f6ac2325640 1 -- 172.21.15.131:0/3592831975 shutdown_connections 2023-09-12T14:54:56.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.657+0000 7f6ac2325640 1 --2- 172.21.15.131:0/3592831975 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6a9c065b20 0x7f6a9c067fe0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:56.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.657+0000 7f6ac2325640 1 --2- 172.21.15.131:0/3592831975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6abc0ab730 0x7f6abc146250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:56.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.657+0000 7f6ac2325640 1 --2- 172.21.15.131:0/3592831975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6abc158180 0x7f6abc14a0d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:56.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.657+0000 7f6ac2325640 1 --2- 172.21.15.131:0/3592831975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6abc06f390 0x7f6abc1478e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:56.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.657+0000 7f6ac2325640 1 -- 172.21.15.131:0/3592831975 >> 172.21.15.131:0/3592831975 conn(0x7f6abc069840 msgr2=0x7f6abc0ac470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:56.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.657+0000 7f6ac2325640 1 -- 172.21.15.131:0/3592831975 shutdown_connections 2023-09-12T14:54:56.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.657+0000 7f6ac2325640 1 -- 172.21.15.131:0/3592831975 wait complete. 2023-09-12T14:54:56.688 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-09-12T14:54:56.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.821+0000 7f9d7dcbe640 1 -- 172.21.15.131:0/772769043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d780cf950 msgr2=0x7f9d780aec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:56.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.821+0000 7f9d7dcbe640 1 --2- 172.21.15.131:0/772769043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d780cf950 0x7f9d780aec00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9d780897e0 tx=0x7f9d7001cca0 comp rx=0 tx=0).stop 2023-09-12T14:54:56.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.821+0000 7f9d7dcbe640 1 -- 172.21.15.131:0/772769043 shutdown_connections 2023-09-12T14:54:56.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.821+0000 7f9d7dcbe640 1 --2- 172.21.15.131:0/772769043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d780cf950 0x7f9d780aec00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:56.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.821+0000 7f9d7dcbe640 1 --2- 172.21.15.131:0/772769043 >> v2:172.21.15.187:3300/0 conn(0x7f9d7815f3f0 0x7f9d780d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:56.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.821+0000 7f9d7dcbe640 1 -- 172.21.15.131:0/772769043 >> 172.21.15.131:0/772769043 conn(0x7f9d7809a210 msgr2=0x7f9d780d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:56.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.821+0000 7f9d7dcbe640 1 -- 172.21.15.131:0/772769043 shutdown_connections 2023-09-12T14:54:56.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.821+0000 7f9d7dcbe640 1 -- 172.21.15.131:0/772769043 wait complete. 2023-09-12T14:54:56.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.825+0000 7f9d7dcbe640 1 Processor -- start 2023-09-12T14:54:56.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.825+0000 7f9d7dcbe640 1 -- start start 2023-09-12T14:54:56.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.825+0000 7f9d7dcbe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d780b2270 0x7f9d780b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:56.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.825+0000 7f9d7dcbe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d780cf950 0x7f9d78156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:56.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.825+0000 7f9d7dcbe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d7815f3f0 0x7f9d780c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:56.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.825+0000 7f9d7dcbe640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d781566b0 con 0x7f9d780b2270 2023-09-12T14:54:56.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.825+0000 7f9d7dcbe640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7815d220 con 0x7f9d7815f3f0 2023-09-12T14:54:56.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.825+0000 7f9d7dcbe640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7815d500 con 0x7f9d780cf950 2023-09-12T14:54:56.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.825+0000 7f9d777fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d780b2270 0x7f9d780b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:56.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.825+0000 7f9d777fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d780b2270 0x7f9d780b0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56206/0 (socket says 172.21.15.131:56206) 2023-09-12T14:54:56.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.825+0000 7f9d777fe640 1 -- 172.21.15.131:0/1923057223 learned_addr learned my addr 172.21.15.131:0/1923057223 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:56.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.825+0000 7f9d77fff640 1 --2- 172.21.15.131:0/1923057223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d7815f3f0 0x7f9d780c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:56.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.825+0000 7f9d6effd640 1 --2- 172.21.15.131:0/1923057223 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d780cf950 0x7f9d78156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:56.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.825+0000 7f9d777fe640 1 -- 172.21.15.131:0/1923057223 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d780cf950 msgr2=0x7f9d78156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:56.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.825+0000 7f9d777fe640 1 --2- 172.21.15.131:0/1923057223 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d780cf950 0x7f9d78156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:56.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.825+0000 7f9d777fe640 1 -- 172.21.15.131:0/1923057223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d7815f3f0 msgr2=0x7f9d780c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:56.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.825+0000 7f9d777fe640 1 --2- 172.21.15.131:0/1923057223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d7815f3f0 0x7f9d780c7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:56.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.825+0000 7f9d777fe640 1 -- 172.21.15.131:0/1923057223 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d700098e0 con 0x7f9d780b2270 2023-09-12T14:54:56.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.825+0000 7f9d777fe640 1 --2- 172.21.15.131:0/1923057223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d780b2270 0x7f9d780b0ea0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800c940 tx=0x7f9d6800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:56.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.825+0000 7f9d74ff9640 1 -- 172.21.15.131:0/1923057223 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d68004220 con 0x7f9d780b2270 2023-09-12T14:54:56.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.825+0000 7f9d74ff9640 1 -- 172.21.15.131:0/1923057223 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9d68004470 con 0x7f9d780b2270 2023-09-12T14:54:56.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.825+0000 7f9d7dcbe640 1 -- 172.21.15.131:0/1923057223 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d78155d10 con 0x7f9d780b2270 2023-09-12T14:54:56.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.825+0000 7f9d74ff9640 1 -- 172.21.15.131:0/1923057223 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d68007bf0 con 0x7f9d780b2270 2023-09-12T14:54:56.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.825+0000 7f9d7dcbe640 1 -- 172.21.15.131:0/1923057223 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d781562d0 con 0x7f9d780b2270 2023-09-12T14:54:56.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.825+0000 7f9d7dcbe640 1 -- 172.21.15.131:0/1923057223 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d78073580 con 0x7f9d780b2270 2023-09-12T14:54:56.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.825+0000 7f9d74ff9640 1 -- 172.21.15.131:0/1923057223 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f9d68007d50 con 0x7f9d780b2270 2023-09-12T14:54:56.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.829+0000 7f9d74ff9640 1 --2- 172.21.15.131:0/1923057223 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9d58065b50 0x7f9d58068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:56.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.829+0000 7f9d74ff9640 1 -- 172.21.15.131:0/1923057223 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f9d68083240 con 0x7f9d780b2270 2023-09-12T14:54:56.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.829+0000 7f9d6effd640 1 --2- 172.21.15.131:0/1923057223 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9d58065b50 0x7f9d58068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:56.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.829+0000 7f9d6effd640 1 --2- 172.21.15.131:0/1923057223 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9d58065b50 0x7f9d58068010 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9d7806afb0 tx=0x7f9d60009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:56.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.833+0000 7f9d74ff9640 1 -- 172.21.15.131:0/1923057223 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f9d68050d70 con 0x7f9d780b2270 2023-09-12T14:54:56.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:56.989+0000 7f9d7dcbe640 1 -- 172.21.15.131:0/1923057223 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9d7809a730 con 0x7f9d780b2270 2023-09-12T14:54:57.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.657+0000 7f9d74ff9640 1 -- 172.21.15.131:0/1923057223 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9d68054390 con 0x7f9d780b2270 2023-09-12T14:54:57.661 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2023-09-12T14:54:57.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.661+0000 7f9d7dcbe640 1 -- 172.21.15.131:0/1923057223 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9d58065b50 msgr2=0x7f9d58068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:57.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.661+0000 7f9d7dcbe640 1 --2- 172.21.15.131:0/1923057223 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9d58065b50 0x7f9d58068010 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9d7806afb0 tx=0x7f9d60009040 comp rx=0 tx=0).stop 2023-09-12T14:54:57.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.661+0000 7f9d7dcbe640 1 -- 172.21.15.131:0/1923057223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d780b2270 msgr2=0x7f9d780b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:57.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.661+0000 7f9d7dcbe640 1 --2- 172.21.15.131:0/1923057223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d780b2270 0x7f9d780b0ea0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800c940 tx=0x7f9d6800ce00 comp rx=0 tx=0).stop 2023-09-12T14:54:57.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.661+0000 7f9d7dcbe640 1 -- 172.21.15.131:0/1923057223 shutdown_connections 2023-09-12T14:54:57.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.661+0000 7f9d7dcbe640 1 --2- 172.21.15.131:0/1923057223 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9d58065b50 0x7f9d58068010 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9d7806afb0 tx=0x7f9d60009040 comp rx=0 tx=0).stop 2023-09-12T14:54:57.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.661+0000 7f9d7dcbe640 1 --2- 172.21.15.131:0/1923057223 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d780cf950 0x7f9d78156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:57.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.661+0000 7f9d7dcbe640 1 --2- 172.21.15.131:0/1923057223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d7815f3f0 0x7f9d780c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:57.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.661+0000 7f9d7dcbe640 1 --2- 172.21.15.131:0/1923057223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d780b2270 0x7f9d780b0ea0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:57.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.661+0000 7f9d7dcbe640 1 -- 172.21.15.131:0/1923057223 >> 172.21.15.131:0/1923057223 conn(0x7f9d7809a210 msgr2=0x7f9d78158160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:57.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.665+0000 7f9d7dcbe640 1 -- 172.21.15.131:0/1923057223 shutdown_connections 2023-09-12T14:54:57.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.665+0000 7f9d7dcbe640 1 -- 172.21.15.131:0/1923057223 wait complete. 2023-09-12T14:54:57.702 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-12T14:54:57.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.829+0000 7f311995e640 1 -- 172.21.15.131:0/2977883036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f311406baa0 msgr2=0x7f3114157970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:57.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.829+0000 7f311995e640 1 --2- 172.21.15.131:0/2977883036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f311406baa0 0x7f3114157970 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3114071150 tx=0x7f310c01d890 comp rx=0 tx=0).stop 2023-09-12T14:54:57.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.829+0000 7f311995e640 1 -- 172.21.15.131:0/2977883036 shutdown_connections 2023-09-12T14:54:57.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.829+0000 7f311995e640 1 --2- 172.21.15.131:0/2977883036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f311406baa0 0x7f3114157970 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:57.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.829+0000 7f311995e640 1 --2- 172.21.15.131:0/2977883036 >> v2:172.21.15.187:3300/0 conn(0x7f3114156e80 0x7f311406eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:57.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.829+0000 7f311995e640 1 -- 172.21.15.131:0/2977883036 >> 172.21.15.131:0/2977883036 conn(0x7f3114069840 msgr2=0x7f31140cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:57.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.829+0000 7f311995e640 1 -- 172.21.15.131:0/2977883036 shutdown_connections 2023-09-12T14:54:57.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.829+0000 7f311995e640 1 -- 172.21.15.131:0/2977883036 wait complete. 2023-09-12T14:54:57.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.829+0000 7f311995e640 1 Processor -- start 2023-09-12T14:54:57.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.833+0000 7f311995e640 1 -- start start 2023-09-12T14:54:57.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.833+0000 7f311995e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f311406baa0 0x7f3114145f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:57.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.833+0000 7f311995e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f311406f1e0 0x7f3114150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:57.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.833+0000 7f311995e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3114156e80 0x7f3114146f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:57.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.833+0000 7f311995e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31141f7ef0 con 0x7f311406baa0 2023-09-12T14:54:57.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.833+0000 7f311995e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31141f81d0 con 0x7f311406f1e0 2023-09-12T14:54:57.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.833+0000 7f311995e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31141f84b0 con 0x7f3114156e80 2023-09-12T14:54:57.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.833+0000 7f3112ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f311406baa0 0x7f3114145f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:57.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.833+0000 7f3112ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f311406baa0 0x7f3114145f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56234/0 (socket says 172.21.15.131:56234) 2023-09-12T14:54:57.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.833+0000 7f3112ffd640 1 -- 172.21.15.131:0/2834357398 learned_addr learned my addr 172.21.15.131:0/2834357398 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:57.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.833+0000 7f3112ffd640 1 -- 172.21.15.131:0/2834357398 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3114156e80 msgr2=0x7f3114146f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:57.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.833+0000 7f31127fc640 1 --2- 172.21.15.131:0/2834357398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f311406f1e0 0x7f3114150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:57.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.833+0000 7f3112ffd640 1 --2- 172.21.15.131:0/2834357398 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3114156e80 0x7f3114146f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:57.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.833+0000 7f3112ffd640 1 -- 172.21.15.131:0/2834357398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f311406f1e0 msgr2=0x7f3114150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:57.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.833+0000 7f3112ffd640 1 --2- 172.21.15.131:0/2834357398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f311406f1e0 0x7f3114150fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:57.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.833+0000 7f3112ffd640 1 -- 172.21.15.131:0/2834357398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310c007450 con 0x7f311406baa0 2023-09-12T14:54:57.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.833+0000 7f3112ffd640 1 --2- 172.21.15.131:0/2834357398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f311406baa0 0x7f3114145f20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f310800d720 tx=0x7f310800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:57.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.833+0000 7f30fbfff640 1 -- 172.21.15.131:0/2834357398 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3108009c30 con 0x7f311406baa0 2023-09-12T14:54:57.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.833+0000 7f30fbfff640 1 -- 172.21.15.131:0/2834357398 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3108009e80 con 0x7f311406baa0 2023-09-12T14:54:57.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.833+0000 7f30fbfff640 1 -- 172.21.15.131:0/2834357398 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3108005260 con 0x7f311406baa0 2023-09-12T14:54:57.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.833+0000 7f311995e640 1 -- 172.21.15.131:0/2834357398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31141f8790 con 0x7f311406baa0 2023-09-12T14:54:57.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.833+0000 7f311995e640 1 -- 172.21.15.131:0/2834357398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3114146460 con 0x7f311406baa0 2023-09-12T14:54:57.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.833+0000 7f311995e640 1 -- 172.21.15.131:0/2834357398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31140cd390 con 0x7f311406baa0 2023-09-12T14:54:57.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.833+0000 7f30fbfff640 1 -- 172.21.15.131:0/2834357398 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f31080053c0 con 0x7f311406baa0 2023-09-12T14:54:57.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.833+0000 7f30fbfff640 1 --2- 172.21.15.131:0/2834357398 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3100065b20 0x7f3100067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:57.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.833+0000 7f31127fc640 1 --2- 172.21.15.131:0/2834357398 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3100065b20 0x7f3100067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:57.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.833+0000 7f30fbfff640 1 -- 172.21.15.131:0/2834357398 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f3108083420 con 0x7f311406baa0 2023-09-12T14:54:57.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.837+0000 7f31127fc640 1 --2- 172.21.15.131:0/2834357398 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3100065b20 0x7f3100067fe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f31140b2880 tx=0x7f3104009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:57.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:57.841+0000 7f30fbfff640 1 -- 172.21.15.131:0/2834357398 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f3108050dd0 con 0x7f311406baa0 2023-09-12T14:54:58.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:58.025+0000 7f311995e640 1 -- 172.21.15.131:0/2834357398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f31141485b0 con 0x7f311406baa0 2023-09-12T14:54:58.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:58.877+0000 7f30fbfff640 1 -- 172.21.15.131:0/2834357398 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f31080200c0 con 0x7f311406baa0 2023-09-12T14:54:58.881 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 2 and data pool 3 2023-09-12T14:54:58.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:58.881+0000 7f311995e640 1 -- 172.21.15.131:0/2834357398 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3100065b20 msgr2=0x7f3100067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:58.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:58.881+0000 7f311995e640 1 --2- 172.21.15.131:0/2834357398 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3100065b20 0x7f3100067fe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f31140b2880 tx=0x7f3104009040 comp rx=0 tx=0).stop 2023-09-12T14:54:58.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:58.881+0000 7f311995e640 1 -- 172.21.15.131:0/2834357398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f311406baa0 msgr2=0x7f3114145f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:58.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:58.881+0000 7f311995e640 1 --2- 172.21.15.131:0/2834357398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f311406baa0 0x7f3114145f20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f310800d720 tx=0x7f310800dbe0 comp rx=0 tx=0).stop 2023-09-12T14:54:58.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:58.881+0000 7f311995e640 1 -- 172.21.15.131:0/2834357398 shutdown_connections 2023-09-12T14:54:58.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:58.881+0000 7f311995e640 1 --2- 172.21.15.131:0/2834357398 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3100065b20 0x7f3100067fe0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:58.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:58.881+0000 7f311995e640 1 --2- 172.21.15.131:0/2834357398 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3114156e80 0x7f3114146f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:58.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:58.881+0000 7f311995e640 1 --2- 172.21.15.131:0/2834357398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f311406f1e0 0x7f3114150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:58.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:58.881+0000 7f311995e640 1 --2- 172.21.15.131:0/2834357398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f311406baa0 0x7f3114145f20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:58.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:58.881+0000 7f311995e640 1 -- 172.21.15.131:0/2834357398 >> 172.21.15.131:0/2834357398 conn(0x7f3114069840 msgr2=0x7f311414de90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:58.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:58.881+0000 7f311995e640 1 -- 172.21.15.131:0/2834357398 shutdown_connections 2023-09-12T14:54:58.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:58.881+0000 7f311995e640 1 -- 172.21.15.131:0/2834357398 wait complete. 2023-09-12T14:54:58.911 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:54:59.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbfd38e640 1 -- 172.21.15.131:0/1467697955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbe0003980 msgr2=0x7fbbe0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:59.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbfd38e640 1 --2- 172.21.15.131:0/1467697955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbe0003980 0x7fbbe0003da0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbbe800bb40 tx=0x7fbbe801cd50 comp rx=0 tx=0).stop 2023-09-12T14:54:59.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbf77fe640 1 -- 172.21.15.131:0/1467697955 reap_dead start 2023-09-12T14:54:59.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbfd38e640 1 -- 172.21.15.131:0/1467697955 shutdown_connections 2023-09-12T14:54:59.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbfd38e640 1 -- 172.21.15.131:0/1467697955 >> 172.21.15.131:0/1467697955 conn(0x7fbbf8069840 msgr2=0x7fbbf80cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:59.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbfd38e640 1 -- 172.21.15.131:0/1467697955 shutdown_connections 2023-09-12T14:54:59.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbfd38e640 1 -- 172.21.15.131:0/1467697955 wait complete. 2023-09-12T14:54:59.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbfd38e640 1 Processor -- start 2023-09-12T14:54:59.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbfd38e640 1 -- start start 2023-09-12T14:54:59.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbfd38e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbe0003980 0x7fbbf814dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:59.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbfd38e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbf81f5c40 0x7fbbf81f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:59.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbfd38e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbf815f7d0 0x7fbbf815fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:59.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbfd38e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf81f6600 con 0x7fbbf815f7d0 2023-09-12T14:54:59.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbfd38e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf8160190 con 0x7fbbf81f5c40 2023-09-12T14:54:59.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbfd38e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf814cf50 con 0x7fbbe0003980 2023-09-12T14:54:59.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbf77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbf815f7d0 0x7fbbf815fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:59.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbf77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbf815f7d0 0x7fbbf815fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38664/0 (socket says 172.21.15.131:38664) 2023-09-12T14:54:59.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbf77fe640 1 -- 172.21.15.131:0/2105394357 learned_addr learned my addr 172.21.15.131:0/2105394357 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:59.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbf6ffd640 1 --2- 172.21.15.131:0/2105394357 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbe0003980 0x7fbbf814dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:59.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbf67fc640 1 --2- 172.21.15.131:0/2105394357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbf81f5c40 0x7fbbf81f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:59.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbf77fe640 1 -- 172.21.15.131:0/2105394357 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbe0003980 msgr2=0x7fbbf814dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:59.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbf77fe640 1 --2- 172.21.15.131:0/2105394357 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbe0003980 0x7fbbf814dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:59.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbf77fe640 1 -- 172.21.15.131:0/2105394357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbf81f5c40 msgr2=0x7fbbf81f60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:59.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbf77fe640 1 --2- 172.21.15.131:0/2105394357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbf81f5c40 0x7fbbf81f60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:59.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbf77fe640 1 -- 172.21.15.131:0/2105394357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf814d230 con 0x7fbbf815f7d0 2023-09-12T14:54:59.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbf77fe640 1 --2- 172.21.15.131:0/2105394357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbf815f7d0 0x7fbbf815fc50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80cd230 tx=0x7fbbf000fed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:59.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbdf7fe640 1 -- 172.21.15.131:0/2105394357 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0003d90 con 0x7fbbf815f7d0 2023-09-12T14:54:59.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbdf7fe640 1 -- 172.21.15.131:0/2105394357 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbbf000d900 con 0x7fbbf815f7d0 2023-09-12T14:54:59.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbfd38e640 1 -- 172.21.15.131:0/2105394357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf814d510 con 0x7fbbf815f7d0 2023-09-12T14:54:59.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbdf7fe640 1 -- 172.21.15.131:0/2105394357 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf000dd10 con 0x7fbbf815f7d0 2023-09-12T14:54:59.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.037+0000 7fbbfd38e640 1 -- 172.21.15.131:0/2105394357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf814dad0 con 0x7fbbf815f7d0 2023-09-12T14:54:59.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.041+0000 7fbbfd38e640 1 -- 172.21.15.131:0/2105394357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf814dcf0 con 0x7fbbf815f7d0 2023-09-12T14:54:59.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.041+0000 7fbbdf7fe640 1 -- 172.21.15.131:0/2105394357 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fbbf0004030 con 0x7fbbf815f7d0 2023-09-12T14:54:59.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.041+0000 7fbbdf7fe640 1 --2- 172.21.15.131:0/2105394357 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fbbe4065b50 0x7fbbe4068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:59.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.041+0000 7fbbdf7fe640 1 -- 172.21.15.131:0/2105394357 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbbf0083b30 con 0x7fbbf815f7d0 2023-09-12T14:54:59.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.041+0000 7fbbf6ffd640 1 --2- 172.21.15.131:0/2105394357 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fbbe4065b50 0x7fbbe4068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:59.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.041+0000 7fbbf6ffd640 1 --2- 172.21.15.131:0/2105394357 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fbbe4065b50 0x7fbbe4068010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80b0000 tx=0x7fbbe80a7040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:59.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.045+0000 7fbbdf7fe640 1 -- 172.21.15.131:0/2105394357 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fbbf0051530 con 0x7fbbf815f7d0 2023-09-12T14:54:59.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.201+0000 7fbbfd38e640 1 -- 172.21.15.131:0/2105394357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbbf8073580 con 0x7fbbf815f7d0 2023-09-12T14:54:59.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.201+0000 7fbbdf7fe640 1 -- 172.21.15.131:0/2105394357 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16554 (secure 0 0 0) 0x7fbbf0054b50 con 0x7fbbf815f7d0 2023-09-12T14:54:59.206 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:54:59.206 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":19,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:54:58.030029+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:54:55.689983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T14:54:57.029447+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:54:59.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.205+0000 7fbbfd38e640 1 -- 172.21.15.131:0/2105394357 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fbbe4065b50 msgr2=0x7fbbe4068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:59.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.205+0000 7fbbfd38e640 1 --2- 172.21.15.131:0/2105394357 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fbbe4065b50 0x7fbbe4068010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80b0000 tx=0x7fbbe80a7040 comp rx=0 tx=0).stop 2023-09-12T14:54:59.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.205+0000 7fbbfd38e640 1 -- 172.21.15.131:0/2105394357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbf815f7d0 msgr2=0x7fbbf815fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:59.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.205+0000 7fbbfd38e640 1 --2- 172.21.15.131:0/2105394357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbf815f7d0 0x7fbbf815fc50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80cd230 tx=0x7fbbf000fed0 comp rx=0 tx=0).stop 2023-09-12T14:54:59.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.205+0000 7fbbfd38e640 1 -- 172.21.15.131:0/2105394357 shutdown_connections 2023-09-12T14:54:59.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.205+0000 7fbbfd38e640 1 --2- 172.21.15.131:0/2105394357 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fbbe4065b50 0x7fbbe4068010 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:59.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.205+0000 7fbbfd38e640 1 --2- 172.21.15.131:0/2105394357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbf815f7d0 0x7fbbf815fc50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:59.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.205+0000 7fbbfd38e640 1 --2- 172.21.15.131:0/2105394357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbf81f5c40 0x7fbbf81f60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:59.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.205+0000 7fbbfd38e640 1 --2- 172.21.15.131:0/2105394357 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbe0003980 0x7fbbf814dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:59.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.205+0000 7fbbfd38e640 1 -- 172.21.15.131:0/2105394357 >> 172.21.15.131:0/2105394357 conn(0x7fbbf8069840 msgr2=0x7fbbf815b620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:59.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.205+0000 7fbbfd38e640 1 -- 172.21.15.131:0/2105394357 shutdown_connections 2023-09-12T14:54:59.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.205+0000 7fbbfd38e640 1 -- 172.21.15.131:0/2105394357 wait complete. 2023-09-12T14:54:59.224 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:54:59.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4eefab640 1 -- 172.21.15.131:0/3280634336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4e8156930 msgr2=0x7fc4e8159dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:59.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4eefab640 1 --2- 172.21.15.131:0/3280634336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4e8156930 0x7fc4e8159dc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc009900 tx=0x7fc4dc01c890 comp rx=0 tx=0).stop 2023-09-12T14:54:59.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4eefab640 1 -- 172.21.15.131:0/3280634336 shutdown_connections 2023-09-12T14:54:59.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4eefab640 1 --2- 172.21.15.131:0/3280634336 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4e80c9370 0x7fc4e80c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:59.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4eefab640 1 --2- 172.21.15.131:0/3280634336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4e8156930 0x7fc4e8159dc0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:59.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4eefab640 1 -- 172.21.15.131:0/3280634336 >> 172.21.15.131:0/3280634336 conn(0x7fc4e8069840 msgr2=0x7fc4e80cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:59.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4eefab640 1 -- 172.21.15.131:0/3280634336 shutdown_connections 2023-09-12T14:54:59.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4eefab640 1 -- 172.21.15.131:0/3280634336 wait complete. 2023-09-12T14:54:59.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4eefab640 1 Processor -- start 2023-09-12T14:54:59.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4eefab640 1 -- start start 2023-09-12T14:54:59.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4eefab640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4e80abb40 0x7fc4e8138390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:59.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4eefab640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4e80c9370 0x7fc4e814dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:59.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4eefab640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4e8156930 0x7fc4e81507f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:59.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4eefab640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e814e6d0 con 0x7fc4e80abb40 2023-09-12T14:54:59.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4eefab640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e814e9b0 con 0x7fc4e80c9370 2023-09-12T14:54:59.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4eefab640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e80ac5e0 con 0x7fc4e8156930 2023-09-12T14:54:59.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4ecd47640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4e80abb40 0x7fc4e8138390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:59.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4ecd47640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4e80abb40 0x7fc4e8138390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38674/0 (socket says 172.21.15.131:38674) 2023-09-12T14:54:59.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4ecd47640 1 -- 172.21.15.131:0/2769795457 learned_addr learned my addr 172.21.15.131:0/2769795457 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:59.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4ecd47640 1 -- 172.21.15.131:0/2769795457 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4e8156930 msgr2=0x7fc4e81507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:59.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4ecd47640 1 --2- 172.21.15.131:0/2769795457 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4e8156930 0x7fc4e81507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:59.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4ecd47640 1 -- 172.21.15.131:0/2769795457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4e80c9370 msgr2=0x7fc4e814dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:59.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4ecd47640 1 --2- 172.21.15.131:0/2769795457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4e80c9370 0x7fc4e814dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:59.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4ecd47640 1 -- 172.21.15.131:0/2769795457 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4dc009560 con 0x7fc4e80abb40 2023-09-12T14:54:59.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4ecd47640 1 --2- 172.21.15.131:0/2769795457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4e80abb40 0x7fc4e8138390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc01a4c0 tx=0x7fc4dc0a5750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:59.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4e57fa640 1 -- 172.21.15.131:0/2769795457 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc005230 con 0x7fc4e80abb40 2023-09-12T14:54:59.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4e57fa640 1 -- 172.21.15.131:0/2769795457 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc4dc0054a0 con 0x7fc4e80abb40 2023-09-12T14:54:59.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4eefab640 1 -- 172.21.15.131:0/2769795457 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e80ac8c0 con 0x7fc4e80abb40 2023-09-12T14:54:59.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.345+0000 7fc4e57fa640 1 -- 172.21.15.131:0/2769795457 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc004930 con 0x7fc4e80abb40 2023-09-12T14:54:59.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.349+0000 7fc4eefab640 1 -- 172.21.15.131:0/2769795457 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e81529e0 con 0x7fc4e80abb40 2023-09-12T14:54:59.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.349+0000 7fc4eefab640 1 -- 172.21.15.131:0/2769795457 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e8152bd0 con 0x7fc4e80abb40 2023-09-12T14:54:59.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.349+0000 7fc4e57fa640 1 -- 172.21.15.131:0/2769795457 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fc4dc0b4070 con 0x7fc4e80abb40 2023-09-12T14:54:59.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.349+0000 7fc4e57fa640 1 --2- 172.21.15.131:0/2769795457 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc4cc065b20 0x7fc4cc067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:59.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.349+0000 7fc4e57fa640 1 -- 172.21.15.131:0/2769795457 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4dc11d020 con 0x7fc4e80abb40 2023-09-12T14:54:59.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.349+0000 7fc4e7fff640 1 --2- 172.21.15.131:0/2769795457 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc4cc065b20 0x7fc4cc067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:59.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.349+0000 7fc4e7fff640 1 --2- 172.21.15.131:0/2769795457 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc4cc065b20 0x7fc4cc067fe0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4000f70 tx=0x7fc4d40079c0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:59.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.353+0000 7fc4e57fa640 1 -- 172.21.15.131:0/2769795457 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fc4dc0ea970 con 0x7fc4e80abb40 2023-09-12T14:54:59.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.509+0000 7fc4eefab640 1 -- 172.21.15.131:0/2769795457 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4e8073580 con 0x7fc4e80abb40 2023-09-12T14:54:59.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.509+0000 7fc4e57fa640 1 -- 172.21.15.131:0/2769795457 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16554 (secure 0 0 0) 0x7fc4dc0a5a00 con 0x7fc4e80abb40 2023-09-12T14:54:59.513 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:54:59.513 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":19,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:54:58.030029+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:54:55.689983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T14:54:57.029447+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:54:59.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.513+0000 7fc4eefab640 1 -- 172.21.15.131:0/2769795457 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc4cc065b20 msgr2=0x7fc4cc067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:59.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.513+0000 7fc4eefab640 1 --2- 172.21.15.131:0/2769795457 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc4cc065b20 0x7fc4cc067fe0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4000f70 tx=0x7fc4d40079c0 comp rx=0 tx=0).stop 2023-09-12T14:54:59.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.513+0000 7fc4eefab640 1 -- 172.21.15.131:0/2769795457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4e80abb40 msgr2=0x7fc4e8138390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:59.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.513+0000 7fc4eefab640 1 --2- 172.21.15.131:0/2769795457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4e80abb40 0x7fc4e8138390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc01a4c0 tx=0x7fc4dc0a5750 comp rx=0 tx=0).stop 2023-09-12T14:54:59.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.513+0000 7fc4eefab640 1 -- 172.21.15.131:0/2769795457 shutdown_connections 2023-09-12T14:54:59.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.513+0000 7fc4eefab640 1 --2- 172.21.15.131:0/2769795457 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc4cc065b20 0x7fc4cc067fe0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:59.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.513+0000 7fc4eefab640 1 --2- 172.21.15.131:0/2769795457 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4e8156930 0x7fc4e81507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:59.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.513+0000 7fc4eefab640 1 --2- 172.21.15.131:0/2769795457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4e80c9370 0x7fc4e814dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:59.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.513+0000 7fc4eefab640 1 --2- 172.21.15.131:0/2769795457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4e80abb40 0x7fc4e8138390 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:59.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.513+0000 7fc4eefab640 1 -- 172.21.15.131:0/2769795457 >> 172.21.15.131:0/2769795457 conn(0x7fc4e8069840 msgr2=0x7fc4e815b520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:59.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.513+0000 7fc4eefab640 1 -- 172.21.15.131:0/2769795457 shutdown_connections 2023-09-12T14:54:59.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.513+0000 7fc4eefab640 1 -- 172.21.15.131:0/2769795457 wait complete. 2023-09-12T14:54:59.532 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-12T14:54:59.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd77c0f9640 1 -- 172.21.15.131:0/599696421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd774158180 msgr2=0x7fd7740cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:59.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd77c0f9640 1 --2- 172.21.15.131:0/599696421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd774158180 0x7fd7740cae70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd770009900 tx=0x7fd77001c890 comp rx=0 tx=0).stop 2023-09-12T14:54:59.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd77c0f9640 1 -- 172.21.15.131:0/599696421 shutdown_connections 2023-09-12T14:54:59.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd77c0f9640 1 --2- 172.21.15.131:0/599696421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd77406b050 0x7fd77406b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:59.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd77c0f9640 1 --2- 172.21.15.131:0/599696421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd774158180 0x7fd7740cae70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:59.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd77c0f9640 1 -- 172.21.15.131:0/599696421 >> 172.21.15.131:0/599696421 conn(0x7fd774069840 msgr2=0x7fd7740cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:59.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd77c0f9640 1 -- 172.21.15.131:0/599696421 shutdown_connections 2023-09-12T14:54:59.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd77c0f9640 1 -- 172.21.15.131:0/599696421 wait complete. 2023-09-12T14:54:59.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd77c0f9640 1 Processor -- start 2023-09-12T14:54:59.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd77c0f9640 1 -- start start 2023-09-12T14:54:59.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd77c0f9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd77406b050 0x7fd7740b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:59.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd77c0f9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7740ab730 0x7fd774147a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:59.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd77c0f9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd774158180 0x7fd77414ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:59.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd77c0f9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7740a64d0 con 0x7fd77406b050 2023-09-12T14:54:59.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd77c0f9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7740a67b0 con 0x7fd774158180 2023-09-12T14:54:59.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd77c0f9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7740a6a90 con 0x7fd7740ab730 2023-09-12T14:54:59.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd779e95640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd77406b050 0x7fd7740b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:59.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd779e95640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd77406b050 0x7fd7740b1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38696/0 (socket says 172.21.15.131:38696) 2023-09-12T14:54:59.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd779e95640 1 -- 172.21.15.131:0/1783353701 learned_addr learned my addr 172.21.15.131:0/1783353701 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:54:59.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd779694640 1 --2- 172.21.15.131:0/1783353701 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7740ab730 0x7fd774147a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:59.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd77a696640 1 --2- 172.21.15.131:0/1783353701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd774158180 0x7fd77414ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:59.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd779e95640 1 -- 172.21.15.131:0/1783353701 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7740ab730 msgr2=0x7fd774147a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:59.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd779e95640 1 --2- 172.21.15.131:0/1783353701 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7740ab730 0x7fd774147a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:59.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd779e95640 1 -- 172.21.15.131:0/1783353701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd774158180 msgr2=0x7fd77414ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:59.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd779e95640 1 --2- 172.21.15.131:0/1783353701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd774158180 0x7fd77414ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:59.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd779e95640 1 -- 172.21.15.131:0/1783353701 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd770009560 con 0x7fd77406b050 2023-09-12T14:54:59.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd779e95640 1 --2- 172.21.15.131:0/1783353701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd77406b050 0x7fd7740b1fa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd76400c500 tx=0x7fd76400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:59.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd76a7fc640 1 -- 172.21.15.131:0/1783353701 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764005220 con 0x7fd77406b050 2023-09-12T14:54:59.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd76a7fc640 1 -- 172.21.15.131:0/1783353701 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd764005470 con 0x7fd77406b050 2023-09-12T14:54:59.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.653+0000 7fd77c0f9640 1 -- 172.21.15.131:0/1783353701 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7740b0b10 con 0x7fd77406b050 2023-09-12T14:54:59.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.657+0000 7fd76a7fc640 1 -- 172.21.15.131:0/1783353701 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764007290 con 0x7fd77406b050 2023-09-12T14:54:59.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.657+0000 7fd77c0f9640 1 -- 172.21.15.131:0/1783353701 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7740b24e0 con 0x7fd77406b050 2023-09-12T14:54:59.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.657+0000 7fd77c0f9640 1 -- 172.21.15.131:0/1783353701 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd774073580 con 0x7fd77406b050 2023-09-12T14:54:59.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.657+0000 7fd76a7fc640 1 -- 172.21.15.131:0/1783353701 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fd7640073f0 con 0x7fd77406b050 2023-09-12T14:54:59.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.657+0000 7fd76a7fc640 1 --2- 172.21.15.131:0/1783353701 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd758065b20 0x7fd758067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:54:59.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.657+0000 7fd779694640 1 --2- 172.21.15.131:0/1783353701 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd758065b20 0x7fd758067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:54:59.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.657+0000 7fd76a7fc640 1 -- 172.21.15.131:0/1783353701 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd764082e50 con 0x7fd77406b050 2023-09-12T14:54:59.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.657+0000 7fd779694640 1 --2- 172.21.15.131:0/1783353701 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd758065b20 0x7fd758067fe0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7700af000 tx=0x7fd77001b0d0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:54:59.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.661+0000 7fd76a7fc640 1 -- 172.21.15.131:0/1783353701 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fd7640507b0 con 0x7fd77406b050 2023-09-12T14:54:59.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.853+0000 7fd77c0f9640 1 -- 172.21.15.131:0/1783353701 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd77406fa10 con 0x7fd77406b050 2023-09-12T14:54:59.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.893+0000 7fd76a7fc640 1 -- 172.21.15.131:0/1783353701 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd764053dd0 con 0x7fd77406b050 2023-09-12T14:54:59.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.893+0000 7fd77c0f9640 1 -- 172.21.15.131:0/1783353701 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd758065b20 msgr2=0x7fd758067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:59.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.893+0000 7fd77c0f9640 1 --2- 172.21.15.131:0/1783353701 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd758065b20 0x7fd758067fe0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7700af000 tx=0x7fd77001b0d0 comp rx=0 tx=0).stop 2023-09-12T14:54:59.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.893+0000 7fd77c0f9640 1 -- 172.21.15.131:0/1783353701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd77406b050 msgr2=0x7fd7740b1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:54:59.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.893+0000 7fd77c0f9640 1 --2- 172.21.15.131:0/1783353701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd77406b050 0x7fd7740b1fa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd76400c500 tx=0x7fd76400c9c0 comp rx=0 tx=0).stop 2023-09-12T14:54:59.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.893+0000 7fd77c0f9640 1 -- 172.21.15.131:0/1783353701 shutdown_connections 2023-09-12T14:54:59.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.893+0000 7fd77c0f9640 1 --2- 172.21.15.131:0/1783353701 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd758065b20 0x7fd758067fe0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:59.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.893+0000 7fd77c0f9640 1 --2- 172.21.15.131:0/1783353701 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7740ab730 0x7fd774147a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:59.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.893+0000 7fd77c0f9640 1 --2- 172.21.15.131:0/1783353701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd774158180 0x7fd77414ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:59.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.893+0000 7fd77c0f9640 1 --2- 172.21.15.131:0/1783353701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd77406b050 0x7fd7740b1fa0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:54:59.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.893+0000 7fd77c0f9640 1 -- 172.21.15.131:0/1783353701 >> 172.21.15.131:0/1783353701 conn(0x7fd774069840 msgr2=0x7fd77414e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:54:59.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.893+0000 7fd77c0f9640 1 -- 172.21.15.131:0/1783353701 shutdown_connections 2023-09-12T14:54:59.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:54:59.893+0000 7fd77c0f9640 1 -- 172.21.15.131:0/1783353701 wait complete. 2023-09-12T14:54:59.912 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-09-12T14:54:59.913 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-09-12T14:55:00.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.033+0000 7fccddc97640 1 -- 172.21.15.131:0/2852244235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccd806d860 msgr2=0x7fccd8069240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:00.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.033+0000 7fccddc97640 1 --2- 172.21.15.131:0/2852244235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccd806d860 0x7fccd8069240 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009900 tx=0x7fccc001c800 comp rx=0 tx=0).stop 2023-09-12T14:55:00.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.033+0000 7fccddc97640 1 -- 172.21.15.131:0/2852244235 shutdown_connections 2023-09-12T14:55:00.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.033+0000 7fccddc97640 1 --2- 172.21.15.131:0/2852244235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fccd8071280 0x7fccd80c8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:00.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.033+0000 7fccddc97640 1 --2- 172.21.15.131:0/2852244235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccd806d860 0x7fccd8069240 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:00.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.033+0000 7fccddc97640 1 -- 172.21.15.131:0/2852244235 >> 172.21.15.131:0/2852244235 conn(0x7fccd806b310 msgr2=0x7fccd806b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:00.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.033+0000 7fccddc97640 1 -- 172.21.15.131:0/2852244235 shutdown_connections 2023-09-12T14:55:00.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.033+0000 7fccddc97640 1 -- 172.21.15.131:0/2852244235 wait complete. 2023-09-12T14:55:00.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fccddc97640 1 Processor -- start 2023-09-12T14:55:00.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fccddc97640 1 -- start start 2023-09-12T14:55:00.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fccddc97640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccd806d860 0x7fccd81480a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:00.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fccddc97640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccd8071280 0x7fccd815b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:00.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fccddc97640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fccd81567d0 0x7fccd8151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:00.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fccddc97640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd8154380 con 0x7fccd8071280 2023-09-12T14:55:00.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fccddc97640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd80cac80 con 0x7fccd806d860 2023-09-12T14:55:00.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fccddc97640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd80caf60 con 0x7fccd81567d0 2023-09-12T14:55:00.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fccd6ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccd8071280 0x7fccd815b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:00.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fccd6ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccd8071280 0x7fccd815b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38710/0 (socket says 172.21.15.131:38710) 2023-09-12T14:55:00.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fccd6ffd640 1 -- 172.21.15.131:0/1310067783 learned_addr learned my addr 172.21.15.131:0/1310067783 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:00.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fccd77fe640 1 --2- 172.21.15.131:0/1310067783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccd806d860 0x7fccd81480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:00.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fccd7fff640 1 --2- 172.21.15.131:0/1310067783 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fccd81567d0 0x7fccd8151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:00.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fccd6ffd640 1 -- 172.21.15.131:0/1310067783 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fccd81567d0 msgr2=0x7fccd8151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:00.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fccd6ffd640 1 --2- 172.21.15.131:0/1310067783 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fccd81567d0 0x7fccd8151860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:00.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fccd6ffd640 1 -- 172.21.15.131:0/1310067783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccd806d860 msgr2=0x7fccd81480a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:00.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fccd6ffd640 1 --2- 172.21.15.131:0/1310067783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccd806d860 0x7fccd81480a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:00.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fccd6ffd640 1 -- 172.21.15.131:0/1310067783 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc0009560 con 0x7fccd8071280 2023-09-12T14:55:00.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fccd7fff640 1 --2- 172.21.15.131:0/1310067783 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fccd81567d0 0x7fccd8151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:00.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fccd6ffd640 1 --2- 172.21.15.131:0/1310067783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccd8071280 0x7fccd815b790 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009850 tx=0x7fccc0005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:00.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fcccffff640 1 -- 172.21.15.131:0/1310067783 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc00a58c0 con 0x7fccd8071280 2023-09-12T14:55:00.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fcccffff640 1 -- 172.21.15.131:0/1310067783 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fccc00038f0 con 0x7fccd8071280 2023-09-12T14:55:00.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fccddc97640 1 -- 172.21.15.131:0/1310067783 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd80cb240 con 0x7fccd8071280 2023-09-12T14:55:00.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fcccffff640 1 -- 172.21.15.131:0/1310067783 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc0003d00 con 0x7fccd8071280 2023-09-12T14:55:00.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fccddc97640 1 -- 172.21.15.131:0/1310067783 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd81485e0 con 0x7fccd8071280 2023-09-12T14:55:00.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fccddc97640 1 -- 172.21.15.131:0/1310067783 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd8073580 con 0x7fccd8071280 2023-09-12T14:55:00.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fcccffff640 1 -- 172.21.15.131:0/1310067783 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fccc00b4040 con 0x7fccd8071280 2023-09-12T14:55:00.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fcccffff640 1 --2- 172.21.15.131:0/1310067783 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fccb8065b50 0x7fccb8068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:00.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fcccffff640 1 -- 172.21.15.131:0/1310067783 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fccc011da60 con 0x7fccd8071280 2023-09-12T14:55:00.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.037+0000 7fccd77fe640 1 --2- 172.21.15.131:0/1310067783 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fccb8065b50 0x7fccb8068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:00.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.041+0000 7fccd77fe640 1 --2- 172.21.15.131:0/1310067783 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fccb8065b50 0x7fccb8068010 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fccc80015b0 tx=0x7fccc8009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:00.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.045+0000 7fcccffff640 1 -- 172.21.15.131:0/1310067783 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fccc00eb440 con 0x7fccd8071280 2023-09-12T14:55:00.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.233+0000 7fccddc97640 1 -- 172.21.15.131:0/1310067783 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fccd8069f70 con 0x7fccd8071280 2023-09-12T14:55:00.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.893+0000 7fcccffff640 1 -- 172.21.15.131:0/1310067783 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7fccc0003e60 con 0x7fccd8071280 2023-09-12T14:55:00.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.897+0000 7fccddc97640 1 -- 172.21.15.131:0/1310067783 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fccb8065b50 msgr2=0x7fccb8068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:00.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.897+0000 7fccddc97640 1 --2- 172.21.15.131:0/1310067783 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fccb8065b50 0x7fccb8068010 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fccc80015b0 tx=0x7fccc8009040 comp rx=0 tx=0).stop 2023-09-12T14:55:00.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.897+0000 7fccddc97640 1 -- 172.21.15.131:0/1310067783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccd8071280 msgr2=0x7fccd815b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:00.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.897+0000 7fccddc97640 1 --2- 172.21.15.131:0/1310067783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccd8071280 0x7fccd815b790 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009850 tx=0x7fccc0005790 comp rx=0 tx=0).stop 2023-09-12T14:55:00.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.897+0000 7fccddc97640 1 -- 172.21.15.131:0/1310067783 shutdown_connections 2023-09-12T14:55:00.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.897+0000 7fccddc97640 1 --2- 172.21.15.131:0/1310067783 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fccb8065b50 0x7fccb8068010 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:00.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.897+0000 7fccddc97640 1 --2- 172.21.15.131:0/1310067783 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fccd81567d0 0x7fccd8151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:00.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.897+0000 7fccddc97640 1 --2- 172.21.15.131:0/1310067783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccd8071280 0x7fccd815b790 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:00.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.897+0000 7fccddc97640 1 --2- 172.21.15.131:0/1310067783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccd806d860 0x7fccd81480a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:00.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.897+0000 7fccddc97640 1 -- 172.21.15.131:0/1310067783 >> 172.21.15.131:0/1310067783 conn(0x7fccd806b310 msgr2=0x7fccd814c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:00.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.897+0000 7fccddc97640 1 -- 172.21.15.131:0/1310067783 shutdown_connections 2023-09-12T14:55:00.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:00.897+0000 7fccddc97640 1 -- 172.21.15.131:0/1310067783 wait complete. 2023-09-12T14:55:00.920 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-09-12T14:55:00.921 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T14:55:01.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.049+0000 7f9e7a881640 1 -- 172.21.15.131:0/818098301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e74142390 msgr2=0x7f9e740b0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:01.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.049+0000 7f9e7a881640 1 --2- 172.21.15.131:0/818098301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e74142390 0x7f9e740b0450 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9e740cbac0 tx=0x7f9e5c01c860 comp rx=0 tx=0).stop 2023-09-12T14:55:01.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.049+0000 7f9e7a881640 1 -- 172.21.15.131:0/818098301 shutdown_connections 2023-09-12T14:55:01.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.049+0000 7f9e7a881640 1 --2- 172.21.15.131:0/818098301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e74142390 0x7f9e740b0450 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:01.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.049+0000 7f9e7a881640 1 --2- 172.21.15.131:0/818098301 >> v2:172.21.15.187:3300/0 conn(0x7f9e74141d20 0x7f9e7415d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:01.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.049+0000 7f9e7a881640 1 -- 172.21.15.131:0/818098301 >> 172.21.15.131:0/818098301 conn(0x7f9e740cdd80 msgr2=0x7f9e740ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:01.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.049+0000 7f9e7a881640 1 -- 172.21.15.131:0/818098301 shutdown_connections 2023-09-12T14:55:01.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.049+0000 7f9e7a881640 1 -- 172.21.15.131:0/818098301 wait complete. 2023-09-12T14:55:01.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.053+0000 7f9e7a881640 1 Processor -- start 2023-09-12T14:55:01.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.053+0000 7f9e7a881640 1 -- start start 2023-09-12T14:55:01.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.053+0000 7f9e7a881640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e740b30c0 0x7f9e741f3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:01.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.053+0000 7f9e7a881640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e74141d20 0x7f9e74159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:01.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.053+0000 7f9e7a881640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e74142390 0x7f9e7413e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:01.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.053+0000 7f9e7a881640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7415df70 con 0x7f9e74142390 2023-09-12T14:55:01.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.053+0000 7f9e7a881640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e74158ce0 con 0x7f9e740b30c0 2023-09-12T14:55:01.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.053+0000 7f9e7a881640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e74158a00 con 0x7f9e74141d20 2023-09-12T14:55:01.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.053+0000 7f9e78e1e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e74142390 0x7f9e7413e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:01.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.053+0000 7f9e78e1e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e74142390 0x7f9e7413e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38720/0 (socket says 172.21.15.131:38720) 2023-09-12T14:55:01.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.053+0000 7f9e78e1e640 1 -- 172.21.15.131:0/1055370298 learned_addr learned my addr 172.21.15.131:0/1055370298 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:01.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.053+0000 7f9e73fff640 1 --2- 172.21.15.131:0/1055370298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e740b30c0 0x7f9e741f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:01.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.053+0000 7f9e6b7fe640 1 --2- 172.21.15.131:0/1055370298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e74141d20 0x7f9e74159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:01.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.053+0000 7f9e78e1e640 1 -- 172.21.15.131:0/1055370298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e74141d20 msgr2=0x7f9e74159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:01.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.053+0000 7f9e78e1e640 1 --2- 172.21.15.131:0/1055370298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e74141d20 0x7f9e74159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:01.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.053+0000 7f9e78e1e640 1 -- 172.21.15.131:0/1055370298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e740b30c0 msgr2=0x7f9e741f3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:01.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.053+0000 7f9e78e1e640 1 --2- 172.21.15.131:0/1055370298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e740b30c0 0x7f9e741f3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:01.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.053+0000 7f9e78e1e640 1 -- 172.21.15.131:0/1055370298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5c009560 con 0x7f9e74142390 2023-09-12T14:55:01.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.053+0000 7f9e6b7fe640 1 --2- 172.21.15.131:0/1055370298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e74141d20 0x7f9e74159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:01.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.053+0000 7f9e78e1e640 1 --2- 172.21.15.131:0/1055370298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e74142390 0x7f9e7413e1f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00c780 tx=0x7f9e6c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:01.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.053+0000 7f9e717fa640 1 -- 172.21.15.131:0/1055370298 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c003d90 con 0x7f9e74142390 2023-09-12T14:55:01.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.053+0000 7f9e717fa640 1 -- 172.21.15.131:0/1055370298 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9e6c00ce90 con 0x7f9e74142390 2023-09-12T14:55:01.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.053+0000 7f9e7a881640 1 -- 172.21.15.131:0/1055370298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e7415d920 con 0x7f9e74142390 2023-09-12T14:55:01.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.053+0000 7f9e717fa640 1 -- 172.21.15.131:0/1055370298 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c0052d0 con 0x7f9e74142390 2023-09-12T14:55:01.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.053+0000 7f9e7a881640 1 -- 172.21.15.131:0/1055370298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e7415a430 con 0x7f9e74142390 2023-09-12T14:55:01.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.053+0000 7f9e7a881640 1 -- 172.21.15.131:0/1055370298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e7415e0e0 con 0x7f9e74142390 2023-09-12T14:55:01.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.053+0000 7f9e717fa640 1 -- 172.21.15.131:0/1055370298 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f9e6c005430 con 0x7f9e74142390 2023-09-12T14:55:01.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.057+0000 7f9e717fa640 1 --2- 172.21.15.131:0/1055370298 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9e54065b20 0x7f9e54067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:01.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.057+0000 7f9e717fa640 1 -- 172.21.15.131:0/1055370298 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e6c082a70 con 0x7f9e74142390 2023-09-12T14:55:01.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.057+0000 7f9e73fff640 1 --2- 172.21.15.131:0/1055370298 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9e54065b20 0x7f9e54067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:01.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.057+0000 7f9e73fff640 1 --2- 172.21.15.131:0/1055370298 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9e54065b20 0x7f9e54067fe0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9e64006fd0 tx=0x7f9e64008040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:01.064 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.061+0000 7f9e717fa640 1 -- 172.21.15.131:0/1055370298 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f9e6c0503c0 con 0x7f9e74142390 2023-09-12T14:55:01.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.249+0000 7f9e7a881640 1 -- 172.21.15.131:0/1055370298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e74073580 con 0x7f9e74142390 2023-09-12T14:55:01.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.249+0000 7f9e717fa640 1 -- 172.21.15.131:0/1055370298 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f9e6c017090 con 0x7f9e74142390 2023-09-12T14:55:01.253 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:01.253 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4344,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/491327908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":491327908},{"type":"v1","addr":"172.21.15.131:6837","nonce":491327908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3470212105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3470212105},{"type":"v1","addr":"172.21.15.187:6833","nonce":3470212105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/91478708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":91478708},{"type":"v1","addr":"172.21.15.131:6836","nonce":91478708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-12T14:54:58.030444+0000","modified":"2023-09-12T14:55:00.236338+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4396":{"gid":4396,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/290727181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":290727181},{"type":"v1","addr":"172.21.15.187:6835","nonce":290727181}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-12T14:55:01.254 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 6 2023-09-12T14:55:01.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.253+0000 7f9e7a881640 1 -- 172.21.15.131:0/1055370298 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9e54065b20 msgr2=0x7f9e54067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:01.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.253+0000 7f9e7a881640 1 --2- 172.21.15.131:0/1055370298 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9e54065b20 0x7f9e54067fe0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9e64006fd0 tx=0x7f9e64008040 comp rx=0 tx=0).stop 2023-09-12T14:55:01.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.253+0000 7f9e7a881640 1 -- 172.21.15.131:0/1055370298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e74142390 msgr2=0x7f9e7413e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:01.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.253+0000 7f9e7a881640 1 --2- 172.21.15.131:0/1055370298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e74142390 0x7f9e7413e1f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00c780 tx=0x7f9e6c00cc40 comp rx=0 tx=0).stop 2023-09-12T14:55:01.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.253+0000 7f9e7a881640 1 -- 172.21.15.131:0/1055370298 shutdown_connections 2023-09-12T14:55:01.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.253+0000 7f9e7a881640 1 --2- 172.21.15.131:0/1055370298 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9e54065b20 0x7f9e54067fe0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:01.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.253+0000 7f9e7a881640 1 --2- 172.21.15.131:0/1055370298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e74141d20 0x7f9e74159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:01.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.253+0000 7f9e7a881640 1 --2- 172.21.15.131:0/1055370298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e74142390 0x7f9e7413e1f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:01.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.253+0000 7f9e7a881640 1 --2- 172.21.15.131:0/1055370298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e740b30c0 0x7f9e741f3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:01.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.253+0000 7f9e7a881640 1 -- 172.21.15.131:0/1055370298 >> 172.21.15.131:0/1055370298 conn(0x7f9e740cdd80 msgr2=0x7f9e7413d590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:01.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.253+0000 7f9e7a881640 1 -- 172.21.15.131:0/1055370298 shutdown_connections 2023-09-12T14:55:01.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.253+0000 7f9e7a881640 1 -- 172.21.15.131:0/1055370298 wait complete. 2023-09-12T14:55:01.271 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:55:01.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.385+0000 7fb7bc56e640 1 -- 172.21.15.131:0/1036086080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7b4156930 msgr2=0x7fb7b4159dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:01.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.385+0000 7fb7bc56e640 1 --2- 172.21.15.131:0/1036086080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7b4156930 0x7fb7b4159dc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb7b406abd0 tx=0x7fb7b001c790 comp rx=0 tx=0).stop 2023-09-12T14:55:01.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.385+0000 7fb7bc56e640 1 -- 172.21.15.131:0/1036086080 shutdown_connections 2023-09-12T14:55:01.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.385+0000 7fb7bc56e640 1 --2- 172.21.15.131:0/1036086080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7b4156930 0x7fb7b4159dc0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:01.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.385+0000 7fb7bc56e640 1 -- 172.21.15.131:0/1036086080 >> 172.21.15.131:0/1036086080 conn(0x7fb7b4069840 msgr2=0x7fb7b40cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:01.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.385+0000 7fb7bc56e640 1 -- 172.21.15.131:0/1036086080 shutdown_connections 2023-09-12T14:55:01.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.385+0000 7fb7bc56e640 1 -- 172.21.15.131:0/1036086080 wait complete. 2023-09-12T14:55:01.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7bc56e640 1 Processor -- start 2023-09-12T14:55:01.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7bc56e640 1 -- start start 2023-09-12T14:55:01.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7bc56e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7b40abb40 0x7fb7b40ac780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:01.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7bc56e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7b40c9370 0x7fb7b414f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:01.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7bc56e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7b4156930 0x7fb7b4148b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:01.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7bc56e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b40ae300 con 0x7fb7b4156930 2023-09-12T14:55:01.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7bc56e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b4149bd0 con 0x7fb7b40c9370 2023-09-12T14:55:01.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7bc56e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b414fb90 con 0x7fb7b40abb40 2023-09-12T14:55:01.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7bab0b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7b4156930 0x7fb7b4148b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:01.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7bab0b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7b4156930 0x7fb7b4148b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38732/0 (socket says 172.21.15.131:38732) 2023-09-12T14:55:01.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7bab0b640 1 -- 172.21.15.131:0/3172520090 learned_addr learned my addr 172.21.15.131:0/3172520090 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:01.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7ba30a640 1 --2- 172.21.15.131:0/3172520090 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7b40abb40 0x7fb7b40ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:01.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7b9b09640 1 --2- 172.21.15.131:0/3172520090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7b40c9370 0x7fb7b414f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:01.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7bab0b640 1 -- 172.21.15.131:0/3172520090 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7b40abb40 msgr2=0x7fb7b40ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:01.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7bab0b640 1 --2- 172.21.15.131:0/3172520090 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7b40abb40 0x7fb7b40ac780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:01.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7bab0b640 1 -- 172.21.15.131:0/3172520090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7b40c9370 msgr2=0x7fb7b414f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:01.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7bab0b640 1 --2- 172.21.15.131:0/3172520090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7b40c9370 0x7fb7b414f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:01.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7bab0b640 1 -- 172.21.15.131:0/3172520090 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b00094a0 con 0x7fb7b4156930 2023-09-12T14:55:01.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7ba30a640 1 --2- 172.21.15.131:0/3172520090 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7b40abb40 0x7fb7b40ac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:01.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7b9b09640 1 --2- 172.21.15.131:0/3172520090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7b40c9370 0x7fb7b414f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:01.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7bab0b640 1 --2- 172.21.15.131:0/3172520090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7b4156930 0x7fb7b4148b50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00cad0 tx=0x7fb7ac00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:01.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7aaffd640 1 -- 172.21.15.131:0/3172520090 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac00aa50 con 0x7fb7b4156930 2023-09-12T14:55:01.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7aaffd640 1 -- 172.21.15.131:0/3172520090 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb7ac00acc0 con 0x7fb7b4156930 2023-09-12T14:55:01.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7bc56e640 1 -- 172.21.15.131:0/3172520090 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b4149090 con 0x7fb7b4156930 2023-09-12T14:55:01.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7aaffd640 1 -- 172.21.15.131:0/3172520090 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac012680 con 0x7fb7b4156930 2023-09-12T14:55:01.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7bc56e640 1 -- 172.21.15.131:0/3172520090 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b4149610 con 0x7fb7b4156930 2023-09-12T14:55:01.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7bc56e640 1 -- 172.21.15.131:0/3172520090 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b40ae470 con 0x7fb7b4156930 2023-09-12T14:55:01.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7aaffd640 1 -- 172.21.15.131:0/3172520090 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fb7ac01b400 con 0x7fb7b4156930 2023-09-12T14:55:01.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7aaffd640 1 --2- 172.21.15.131:0/3172520090 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb798065b20 0x7fb798067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:01.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7aaffd640 1 -- 172.21.15.131:0/3172520090 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb7ac0838f0 con 0x7fb7b4156930 2023-09-12T14:55:01.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.389+0000 7fb7ba30a640 1 --2- 172.21.15.131:0/3172520090 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb798065b20 0x7fb798067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:01.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.393+0000 7fb7ba30a640 1 --2- 172.21.15.131:0/3172520090 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb798065b20 0x7fb798067fe0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00af000 tx=0x7fb7b001afd0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:01.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.393+0000 7fb7aaffd640 1 -- 172.21.15.131:0/3172520090 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fb7ac0512d0 con 0x7fb7b4156930 2023-09-12T14:55:01.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.537+0000 7fb7bc56e640 1 -- 172.21.15.131:0/3172520090 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7b40b1b60 con 0x7fb7b4156930 2023-09-12T14:55:01.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.541+0000 7fb7aaffd640 1 -- 172.21.15.131:0/3172520090 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fb7ac016090 con 0x7fb7b4156930 2023-09-12T14:55:01.543 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:01.544 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:54:59.694778+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:54:55.689983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T14:54:57.029447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:55:01.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.541+0000 7fb7bc56e640 1 -- 172.21.15.131:0/3172520090 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb798065b20 msgr2=0x7fb798067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:01.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.541+0000 7fb7bc56e640 1 --2- 172.21.15.131:0/3172520090 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb798065b20 0x7fb798067fe0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00af000 tx=0x7fb7b001afd0 comp rx=0 tx=0).stop 2023-09-12T14:55:01.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.541+0000 7fb7bc56e640 1 -- 172.21.15.131:0/3172520090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7b4156930 msgr2=0x7fb7b4148b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:01.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.541+0000 7fb7bc56e640 1 --2- 172.21.15.131:0/3172520090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7b4156930 0x7fb7b4148b50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00cad0 tx=0x7fb7ac00cbc0 comp rx=0 tx=0).stop 2023-09-12T14:55:01.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.541+0000 7fb7bc56e640 1 -- 172.21.15.131:0/3172520090 shutdown_connections 2023-09-12T14:55:01.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.541+0000 7fb7bc56e640 1 --2- 172.21.15.131:0/3172520090 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb798065b20 0x7fb798067fe0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:01.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.541+0000 7fb7bc56e640 1 --2- 172.21.15.131:0/3172520090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7b4156930 0x7fb7b4148b50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:01.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.541+0000 7fb7bc56e640 1 --2- 172.21.15.131:0/3172520090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7b40c9370 0x7fb7b414f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:01.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.541+0000 7fb7bc56e640 1 --2- 172.21.15.131:0/3172520090 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7b40abb40 0x7fb7b40ac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:01.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.541+0000 7fb7bc56e640 1 -- 172.21.15.131:0/3172520090 >> 172.21.15.131:0/3172520090 conn(0x7fb7b4069840 msgr2=0x7fb7b414e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:01.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.541+0000 7fb7bc56e640 1 -- 172.21.15.131:0/3172520090 shutdown_connections 2023-09-12T14:55:01.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.541+0000 7fb7bc56e640 1 -- 172.21.15.131:0/3172520090 wait complete. 2023-09-12T14:55:01.560 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-12T14:55:01.561 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-12T14:55:01.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb0804e4640 1 -- 172.21.15.131:0/710317858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0780c9370 msgr2=0x7fb0780c9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:01.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb0804e4640 1 --2- 172.21.15.131:0/710317858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0780c9370 0x7fb0780c9780 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb07806d170 tx=0x7fb07001cd30 comp rx=0 tx=0).stop 2023-09-12T14:55:01.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb0804e4640 1 -- 172.21.15.131:0/710317858 shutdown_connections 2023-09-12T14:55:01.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb0804e4640 1 --2- 172.21.15.131:0/710317858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0780c9370 0x7fb0780c9780 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:01.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb0804e4640 1 --2- 172.21.15.131:0/710317858 >> v2:172.21.15.187:3300/0 conn(0x7fb0780abb40 0x7fb07806b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:01.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb0804e4640 1 -- 172.21.15.131:0/710317858 >> 172.21.15.131:0/710317858 conn(0x7fb078069840 msgr2=0x7fb0780cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:01.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb0804e4640 1 -- 172.21.15.131:0/710317858 shutdown_connections 2023-09-12T14:55:01.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb0804e4640 1 -- 172.21.15.131:0/710317858 wait complete. 2023-09-12T14:55:01.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb0804e4640 1 Processor -- start 2023-09-12T14:55:01.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb0804e4640 1 -- start start 2023-09-12T14:55:01.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb0804e4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0780abb40 0x7fb0781460f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:01.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb0804e4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0780c9370 0x7fb07814bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:01.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb0804e4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb078156930 0x7fb078138060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:01.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb0804e4640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07814c080 con 0x7fb0780c9370 2023-09-12T14:55:01.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb0804e4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb078146780 con 0x7fb078156930 2023-09-12T14:55:01.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb0804e4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb078146a60 con 0x7fb0780abb40 2023-09-12T14:55:01.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb07da7f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0780c9370 0x7fb07814bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:01.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb07da7f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0780c9370 0x7fb07814bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38752/0 (socket says 172.21.15.131:38752) 2023-09-12T14:55:01.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb07da7f640 1 -- 172.21.15.131:0/1381513422 learned_addr learned my addr 172.21.15.131:0/1381513422 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:01.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb07da7f640 1 -- 172.21.15.131:0/1381513422 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0780abb40 msgr2=0x7fb0781460f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:01.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb07da7f640 1 --2- 172.21.15.131:0/1381513422 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0780abb40 0x7fb0781460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:01.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb07ea81640 1 --2- 172.21.15.131:0/1381513422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb078156930 0x7fb078138060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:01.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb07da7f640 1 -- 172.21.15.131:0/1381513422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb078156930 msgr2=0x7fb078138060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:01.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb07da7f640 1 --2- 172.21.15.131:0/1381513422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb078156930 0x7fb078138060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:01.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb07da7f640 1 -- 172.21.15.131:0/1381513422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0700098e0 con 0x7fb0780c9370 2023-09-12T14:55:01.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb07da7f640 1 --2- 172.21.15.131:0/1381513422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0780c9370 0x7fb07814bb40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb07400d950 tx=0x7fb07400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:01.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb06effd640 1 -- 172.21.15.131:0/1381513422 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb074009c30 con 0x7fb0780c9370 2023-09-12T14:55:01.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb06effd640 1 -- 172.21.15.131:0/1381513422 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb074009e80 con 0x7fb0780c9370 2023-09-12T14:55:01.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb0804e4640 1 -- 172.21.15.131:0/1381513422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb078146d40 con 0x7fb0780c9370 2023-09-12T14:55:01.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb06effd640 1 -- 172.21.15.131:0/1381513422 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0740106c0 con 0x7fb0780c9370 2023-09-12T14:55:01.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.681+0000 7fb0804e4640 1 -- 172.21.15.131:0/1381513422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb078147300 con 0x7fb0780c9370 2023-09-12T14:55:01.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.685+0000 7fb0804e4640 1 -- 172.21.15.131:0/1381513422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb078073580 con 0x7fb0780c9370 2023-09-12T14:55:01.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.685+0000 7fb06effd640 1 -- 172.21.15.131:0/1381513422 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fb074010820 con 0x7fb0780c9370 2023-09-12T14:55:01.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.685+0000 7fb06effd640 1 --2- 172.21.15.131:0/1381513422 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb064065b50 0x7fb064068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:01.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.685+0000 7fb06effd640 1 -- 172.21.15.131:0/1381513422 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb074083700 con 0x7fb0780c9370 2023-09-12T14:55:01.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.685+0000 7fb07e280640 1 --2- 172.21.15.131:0/1381513422 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb064065b50 0x7fb064068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:01.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.685+0000 7fb07e280640 1 --2- 172.21.15.131:0/1381513422 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb064065b50 0x7fb064068010 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb068000f70 tx=0x7fb0680079e0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:01.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.689+0000 7fb06effd640 1 -- 172.21.15.131:0/1381513422 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fb0740510e0 con 0x7fb0780c9370 2023-09-12T14:55:01.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.837+0000 7fb0804e4640 1 -- 172.21.15.131:0/1381513422 --> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb0780cd4d0 con 0x7fb064065b50 2023-09-12T14:55:01.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.965+0000 7fb06effd640 1 -- 172.21.15.131:0/1381513422 <== mgr.4200 v2:172.21.15.131:6832/2298103651 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264611 (secure 0 0 0) 0x7fb0780cd4d0 con 0x7fb064065b50 2023-09-12T14:55:01.969 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:01.971 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-09-12T14:55:01.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.969+0000 7fb0804e4640 1 -- 172.21.15.131:0/1381513422 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb064065b50 msgr2=0x7fb064068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:01.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.969+0000 7fb0804e4640 1 --2- 172.21.15.131:0/1381513422 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb064065b50 0x7fb064068010 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb068000f70 tx=0x7fb0680079e0 comp rx=0 tx=0).stop 2023-09-12T14:55:01.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.969+0000 7fb0804e4640 1 -- 172.21.15.131:0/1381513422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0780c9370 msgr2=0x7fb07814bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:01.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.969+0000 7fb0804e4640 1 --2- 172.21.15.131:0/1381513422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0780c9370 0x7fb07814bb40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb07400d950 tx=0x7fb07400de10 comp rx=0 tx=0).stop 2023-09-12T14:55:01.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.969+0000 7fb0804e4640 1 -- 172.21.15.131:0/1381513422 shutdown_connections 2023-09-12T14:55:01.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.969+0000 7fb0804e4640 1 --2- 172.21.15.131:0/1381513422 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb064065b50 0x7fb064068010 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:01.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.969+0000 7fb0804e4640 1 --2- 172.21.15.131:0/1381513422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb078156930 0x7fb078138060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:01.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.969+0000 7fb0804e4640 1 --2- 172.21.15.131:0/1381513422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0780c9370 0x7fb07814bb40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:01.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.969+0000 7fb0804e4640 1 --2- 172.21.15.131:0/1381513422 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0780abb40 0x7fb0781460f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:01.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.969+0000 7fb0804e4640 1 -- 172.21.15.131:0/1381513422 >> 172.21.15.131:0/1381513422 conn(0x7fb078069840 msgr2=0x7fb0780cca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:01.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.969+0000 7fb0804e4640 1 -- 172.21.15.131:0/1381513422 shutdown_connections 2023-09-12T14:55:01.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:01.969+0000 7fb0804e4640 1 -- 172.21.15.131:0/1381513422 wait complete. 2023-09-12T14:55:01.989 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-09-12T14:55:01.580348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":144,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218980,"kb_used_data":2796,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755740,"statfs":{"total":773094113280,"available":772869877760,"internally_reserved":0,"allocated":2863104,"data_stored":1739121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":63,"apply_latency_ms":63,"commit_latency_ns":63000000,"apply_latency_ns":63000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.923057"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806828+0000","last_change":"2023-09-12T14:54:58.747316+0000","last_active":"2023-09-12T14:54:59.806828+0000","last_peered":"2023-09-12T14:54:59.806828+0000","last_clean":"2023-09-12T14:54:59.806828+0000","last_became_active":"2023-09-12T14:54:58.746402+0000","last_became_peered":"2023-09-12T14:54:58.746402+0000","last_unstale":"2023-09-12T14:54:59.806828+0000","last_undegraded":"2023-09-12T14:54:59.806828+0000","last_fullsized":"2023-09-12T14:54:59.806828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:26:45.799536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.701256+0000","last_change":"2023-09-12T14:54:58.726405+0000","last_active":"2023-09-12T14:54:59.701256+0000","last_peered":"2023-09-12T14:54:59.701256+0000","last_clean":"2023-09-12T14:54:59.701256+0000","last_became_active":"2023-09-12T14:54:58.723598+0000","last_became_peered":"2023-09-12T14:54:58.723598+0000","last_unstale":"2023-09-12T14:54:59.701256+0000","last_undegraded":"2023-09-12T14:54:59.701256+0000","last_fullsized":"2023-09-12T14:54:59.701256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:56:13.426275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806975+0000","last_change":"2023-09-12T14:54:58.735800+0000","last_active":"2023-09-12T14:54:59.806975+0000","last_peered":"2023-09-12T14:54:59.806975+0000","last_clean":"2023-09-12T14:54:59.806975+0000","last_became_active":"2023-09-12T14:54:58.734886+0000","last_became_peered":"2023-09-12T14:54:58.734886+0000","last_unstale":"2023-09-12T14:54:59.806975+0000","last_undegraded":"2023-09-12T14:54:59.806975+0000","last_fullsized":"2023-09-12T14:54:59.806975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:56:13.426275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.705851+0000","last_change":"2023-09-12T14:54:58.751668+0000","last_active":"2023-09-12T14:54:59.705851+0000","last_peered":"2023-09-12T14:54:59.705851+0000","last_clean":"2023-09-12T14:54:59.705851+0000","last_became_active":"2023-09-12T14:54:58.751016+0000","last_became_peered":"2023-09-12T14:54:58.751016+0000","last_unstale":"2023-09-12T14:54:59.705851+0000","last_undegraded":"2023-09-12T14:54:59.705851+0000","last_fullsized":"2023-09-12T14:54:59.705851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:55:04.268808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806474+0000","last_change":"2023-09-12T14:54:58.749393+0000","last_active":"2023-09-12T14:54:59.806474+0000","last_peered":"2023-09-12T14:54:59.806474+0000","last_clean":"2023-09-12T14:54:59.806474+0000","last_became_active":"2023-09-12T14:54:58.748795+0000","last_became_peered":"2023-09-12T14:54:58.748795+0000","last_unstale":"2023-09-12T14:54:59.806474+0000","last_undegraded":"2023-09-12T14:54:59.806474+0000","last_fullsized":"2023-09-12T14:54:59.806474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:18:31.201019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.705919+0000","last_change":"2023-09-12T14:54:58.722071+0000","last_active":"2023-09-12T14:54:59.705919+0000","last_peered":"2023-09-12T14:54:59.705919+0000","last_clean":"2023-09-12T14:54:59.705919+0000","last_became_active":"2023-09-12T14:54:58.721356+0000","last_became_peered":"2023-09-12T14:54:58.721356+0000","last_unstale":"2023-09-12T14:54:59.705919+0000","last_undegraded":"2023-09-12T14:54:59.705919+0000","last_fullsized":"2023-09-12T14:54:59.705919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:26:45.799536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806581+0000","last_change":"2023-09-12T14:54:58.762364+0000","last_active":"2023-09-12T14:54:59.806581+0000","last_peered":"2023-09-12T14:54:59.806581+0000","last_clean":"2023-09-12T14:54:59.806581+0000","last_became_active":"2023-09-12T14:54:58.761809+0000","last_became_peered":"2023-09-12T14:54:58.761809+0000","last_unstale":"2023-09-12T14:54:59.806581+0000","last_undegraded":"2023-09-12T14:54:59.806581+0000","last_fullsized":"2023-09-12T14:54:59.806581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:55:04.268808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807929+0000","last_change":"2023-09-12T14:54:58.749962+0000","last_active":"2023-09-12T14:54:59.807929+0000","last_peered":"2023-09-12T14:54:59.807929+0000","last_clean":"2023-09-12T14:54:59.807929+0000","last_became_active":"2023-09-12T14:54:58.749252+0000","last_became_peered":"2023-09-12T14:54:58.749252+0000","last_unstale":"2023-09-12T14:54:59.807929+0000","last_undegraded":"2023-09-12T14:54:59.807929+0000","last_fullsized":"2023-09-12T14:54:59.807929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:58:08.195627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807390+0000","last_change":"2023-09-12T14:54:58.745069+0000","last_active":"2023-09-12T14:54:59.807390+0000","last_peered":"2023-09-12T14:54:59.807390+0000","last_clean":"2023-09-12T14:54:59.807390+0000","last_became_active":"2023-09-12T14:54:58.744344+0000","last_became_peered":"2023-09-12T14:54:58.744344+0000","last_unstale":"2023-09-12T14:54:59.807390+0000","last_undegraded":"2023-09-12T14:54:59.807390+0000","last_fullsized":"2023-09-12T14:54:59.807390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:07:30.650142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807913+0000","last_change":"2023-09-12T14:54:58.746751+0000","last_active":"2023-09-12T14:54:59.807913+0000","last_peered":"2023-09-12T14:54:59.807913+0000","last_clean":"2023-09-12T14:54:59.807913+0000","last_became_active":"2023-09-12T14:54:58.746118+0000","last_became_peered":"2023-09-12T14:54:58.746118+0000","last_unstale":"2023-09-12T14:54:59.807913+0000","last_undegraded":"2023-09-12T14:54:59.807913+0000","last_fullsized":"2023-09-12T14:54:59.807913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:18:31.201019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807341+0000","last_change":"2023-09-12T14:54:58.744115+0000","last_active":"2023-09-12T14:54:59.807341+0000","last_peered":"2023-09-12T14:54:59.807341+0000","last_clean":"2023-09-12T14:54:59.807341+0000","last_became_active":"2023-09-12T14:54:58.743477+0000","last_became_peered":"2023-09-12T14:54:58.743477+0000","last_unstale":"2023-09-12T14:54:59.807341+0000","last_undegraded":"2023-09-12T14:54:59.807341+0000","last_fullsized":"2023-09-12T14:54:59.807341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:33.647832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.709077+0000","last_change":"2023-09-12T14:54:58.750301+0000","last_active":"2023-09-12T14:54:59.709077+0000","last_peered":"2023-09-12T14:54:59.709077+0000","last_clean":"2023-09-12T14:54:59.709077+0000","last_became_active":"2023-09-12T14:54:58.748251+0000","last_became_peered":"2023-09-12T14:54:58.748251+0000","last_unstale":"2023-09-12T14:54:59.709077+0000","last_undegraded":"2023-09-12T14:54:59.709077+0000","last_fullsized":"2023-09-12T14:54:59.709077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:35.965864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808454+0000","last_change":"2023-09-12T14:54:58.750934+0000","last_active":"2023-09-12T14:54:59.808454+0000","last_peered":"2023-09-12T14:54:59.808454+0000","last_clean":"2023-09-12T14:54:59.808454+0000","last_became_active":"2023-09-12T14:54:58.750202+0000","last_became_peered":"2023-09-12T14:54:58.750202+0000","last_unstale":"2023-09-12T14:54:59.808454+0000","last_undegraded":"2023-09-12T14:54:59.808454+0000","last_fullsized":"2023-09-12T14:54:59.808454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:33.647832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806666+0000","last_change":"2023-09-12T14:54:58.730838+0000","last_active":"2023-09-12T14:54:59.806666+0000","last_peered":"2023-09-12T14:54:59.806666+0000","last_clean":"2023-09-12T14:54:59.806666+0000","last_became_active":"2023-09-12T14:54:58.727571+0000","last_became_peered":"2023-09-12T14:54:58.727571+0000","last_unstale":"2023-09-12T14:54:59.806666+0000","last_undegraded":"2023-09-12T14:54:59.806666+0000","last_fullsized":"2023-09-12T14:54:59.806666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:58:08.195627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.700662+0000","last_change":"2023-09-12T14:54:58.723535+0000","last_active":"2023-09-12T14:54:59.700662+0000","last_peered":"2023-09-12T14:54:59.700662+0000","last_clean":"2023-09-12T14:54:59.700662+0000","last_became_active":"2023-09-12T14:54:58.722775+0000","last_became_peered":"2023-09-12T14:54:58.722775+0000","last_unstale":"2023-09-12T14:54:59.700662+0000","last_undegraded":"2023-09-12T14:54:59.700662+0000","last_fullsized":"2023-09-12T14:54:59.700662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:54:14.557968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.705388+0000","last_change":"2023-09-12T14:54:58.720670+0000","last_active":"2023-09-12T14:54:59.705388+0000","last_peered":"2023-09-12T14:54:59.705388+0000","last_clean":"2023-09-12T14:54:59.705388+0000","last_became_active":"2023-09-12T14:54:58.719787+0000","last_became_peered":"2023-09-12T14:54:58.719787+0000","last_unstale":"2023-09-12T14:54:59.705388+0000","last_undegraded":"2023-09-12T14:54:59.705388+0000","last_fullsized":"2023-09-12T14:54:59.705388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:33:30.961102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.708175+0000","last_change":"2023-09-12T14:54:58.733183+0000","last_active":"2023-09-12T14:54:59.708175+0000","last_peered":"2023-09-12T14:54:59.708175+0000","last_clean":"2023-09-12T14:54:59.708175+0000","last_became_active":"2023-09-12T14:54:58.732383+0000","last_became_peered":"2023-09-12T14:54:58.732383+0000","last_unstale":"2023-09-12T14:54:59.708175+0000","last_undegraded":"2023-09-12T14:54:59.708175+0000","last_fullsized":"2023-09-12T14:54:59.708175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:53:00.606916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805991+0000","last_change":"2023-09-12T14:54:58.751801+0000","last_active":"2023-09-12T14:54:59.805991+0000","last_peered":"2023-09-12T14:54:59.805991+0000","last_clean":"2023-09-12T14:54:59.805991+0000","last_became_active":"2023-09-12T14:54:58.748758+0000","last_became_peered":"2023-09-12T14:54:58.748758+0000","last_unstale":"2023-09-12T14:54:59.805991+0000","last_undegraded":"2023-09-12T14:54:59.805991+0000","last_fullsized":"2023-09-12T14:54:59.805991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:34:39.391000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807256+0000","last_change":"2023-09-12T14:54:58.745955+0000","last_active":"2023-09-12T14:54:59.807256+0000","last_peered":"2023-09-12T14:54:59.807256+0000","last_clean":"2023-09-12T14:54:59.807256+0000","last_became_active":"2023-09-12T14:54:58.744876+0000","last_became_peered":"2023-09-12T14:54:58.744876+0000","last_unstale":"2023-09-12T14:54:59.807256+0000","last_undegraded":"2023-09-12T14:54:59.807256+0000","last_fullsized":"2023-09-12T14:54:59.807256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:13:39.293184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807685+0000","last_change":"2023-09-12T14:54:58.744120+0000","last_active":"2023-09-12T14:54:59.807685+0000","last_peered":"2023-09-12T14:54:59.807685+0000","last_clean":"2023-09-12T14:54:59.807685+0000","last_became_active":"2023-09-12T14:54:58.743538+0000","last_became_peered":"2023-09-12T14:54:58.743538+0000","last_unstale":"2023-09-12T14:54:59.807685+0000","last_undegraded":"2023-09-12T14:54:59.807685+0000","last_fullsized":"2023-09-12T14:54:59.807685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:10:09.339305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806423+0000","last_change":"2023-09-12T14:54:58.733606+0000","last_active":"2023-09-12T14:54:59.806423+0000","last_peered":"2023-09-12T14:54:59.806423+0000","last_clean":"2023-09-12T14:54:59.806423+0000","last_became_active":"2023-09-12T14:54:58.732811+0000","last_became_peered":"2023-09-12T14:54:58.732811+0000","last_unstale":"2023-09-12T14:54:59.806423+0000","last_undegraded":"2023-09-12T14:54:59.806423+0000","last_fullsized":"2023-09-12T14:54:59.806423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:25:02.224632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706033+0000","last_change":"2023-09-12T14:54:58.733719+0000","last_active":"2023-09-12T14:54:59.706033+0000","last_peered":"2023-09-12T14:54:59.706033+0000","last_clean":"2023-09-12T14:54:59.706033+0000","last_became_active":"2023-09-12T14:54:58.733053+0000","last_became_peered":"2023-09-12T14:54:58.733053+0000","last_unstale":"2023-09-12T14:54:59.706033+0000","last_undegraded":"2023-09-12T14:54:59.706033+0000","last_fullsized":"2023-09-12T14:54:59.706033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:16:18.487363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808066+0000","last_change":"2023-09-12T14:54:58.744857+0000","last_active":"2023-09-12T14:54:59.808066+0000","last_peered":"2023-09-12T14:54:59.808066+0000","last_clean":"2023-09-12T14:54:59.808066+0000","last_became_active":"2023-09-12T14:54:58.744250+0000","last_became_peered":"2023-09-12T14:54:58.744250+0000","last_unstale":"2023-09-12T14:54:59.808066+0000","last_undegraded":"2023-09-12T14:54:59.808066+0000","last_fullsized":"2023-09-12T14:54:59.808066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:09:24.488408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.705573+0000","last_change":"2023-09-12T14:54:58.751371+0000","last_active":"2023-09-12T14:54:59.705573+0000","last_peered":"2023-09-12T14:54:59.705573+0000","last_clean":"2023-09-12T14:54:59.705573+0000","last_became_active":"2023-09-12T14:54:58.750585+0000","last_became_peered":"2023-09-12T14:54:58.750585+0000","last_unstale":"2023-09-12T14:54:59.705573+0000","last_undegraded":"2023-09-12T14:54:59.705573+0000","last_fullsized":"2023-09-12T14:54:59.705573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:18:31.201019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.809302+0000","last_change":"2023-09-12T14:54:58.745970+0000","last_active":"2023-09-12T14:54:59.809302+0000","last_peered":"2023-09-12T14:54:59.809302+0000","last_clean":"2023-09-12T14:54:59.809302+0000","last_became_active":"2023-09-12T14:54:58.745163+0000","last_became_peered":"2023-09-12T14:54:58.745163+0000","last_unstale":"2023-09-12T14:54:59.809302+0000","last_undegraded":"2023-09-12T14:54:59.809302+0000","last_fullsized":"2023-09-12T14:54:59.809302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:55:04.268808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807739+0000","last_change":"2023-09-12T14:54:58.746782+0000","last_active":"2023-09-12T14:54:59.807739+0000","last_peered":"2023-09-12T14:54:59.807739+0000","last_clean":"2023-09-12T14:54:59.807739+0000","last_became_active":"2023-09-12T14:54:58.746149+0000","last_became_peered":"2023-09-12T14:54:58.746149+0000","last_unstale":"2023-09-12T14:54:59.807739+0000","last_undegraded":"2023-09-12T14:54:59.807739+0000","last_fullsized":"2023-09-12T14:54:59.807739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:33:30.961102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.701439+0000","last_change":"2023-09-12T14:54:58.747221+0000","last_active":"2023-09-12T14:54:59.701439+0000","last_peered":"2023-09-12T14:54:59.701439+0000","last_clean":"2023-09-12T14:54:59.701439+0000","last_became_active":"2023-09-12T14:54:58.746509+0000","last_became_peered":"2023-09-12T14:54:58.746509+0000","last_unstale":"2023-09-12T14:54:59.701439+0000","last_undegraded":"2023-09-12T14:54:59.701439+0000","last_fullsized":"2023-09-12T14:54:59.701439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:33.647832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808902+0000","last_change":"2023-09-12T14:54:58.754523+0000","last_active":"2023-09-12T14:54:59.808902+0000","last_peered":"2023-09-12T14:54:59.808902+0000","last_clean":"2023-09-12T14:54:59.808902+0000","last_became_active":"2023-09-12T14:54:58.753849+0000","last_became_peered":"2023-09-12T14:54:58.753849+0000","last_unstale":"2023-09-12T14:54:59.808902+0000","last_undegraded":"2023-09-12T14:54:59.808902+0000","last_fullsized":"2023-09-12T14:54:59.808902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:18:31.201019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806568+0000","last_change":"2023-09-12T14:54:58.750572+0000","last_active":"2023-09-12T14:54:59.806568+0000","last_peered":"2023-09-12T14:54:59.806568+0000","last_clean":"2023-09-12T14:54:59.806568+0000","last_became_active":"2023-09-12T14:54:58.750004+0000","last_became_peered":"2023-09-12T14:54:58.750004+0000","last_unstale":"2023-09-12T14:54:59.806568+0000","last_undegraded":"2023-09-12T14:54:59.806568+0000","last_fullsized":"2023-09-12T14:54:59.806568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:54:14.557968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706258+0000","last_change":"2023-09-12T14:54:58.754746+0000","last_active":"2023-09-12T14:54:59.706258+0000","last_peered":"2023-09-12T14:54:59.706258+0000","last_clean":"2023-09-12T14:54:59.706258+0000","last_became_active":"2023-09-12T14:54:58.754081+0000","last_became_peered":"2023-09-12T14:54:58.754081+0000","last_unstale":"2023-09-12T14:54:59.706258+0000","last_undegraded":"2023-09-12T14:54:59.706258+0000","last_fullsized":"2023-09-12T14:54:59.706258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:23:41.038660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805117+0000","last_change":"2023-09-12T14:54:58.739330+0000","last_active":"2023-09-12T14:54:59.805117+0000","last_peered":"2023-09-12T14:54:59.805117+0000","last_clean":"2023-09-12T14:54:59.805117+0000","last_became_active":"2023-09-12T14:54:58.738451+0000","last_became_peered":"2023-09-12T14:54:58.738451+0000","last_unstale":"2023-09-12T14:54:59.805117+0000","last_undegraded":"2023-09-12T14:54:59.805117+0000","last_fullsized":"2023-09-12T14:54:59.805117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:35.965864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805898+0000","last_change":"2023-09-12T14:54:58.745424+0000","last_active":"2023-09-12T14:54:59.805898+0000","last_peered":"2023-09-12T14:54:59.805898+0000","last_clean":"2023-09-12T14:54:59.805898+0000","last_became_active":"2023-09-12T14:54:58.744842+0000","last_became_peered":"2023-09-12T14:54:58.744842+0000","last_unstale":"2023-09-12T14:54:59.805898+0000","last_undegraded":"2023-09-12T14:54:59.805898+0000","last_fullsized":"2023-09-12T14:54:59.805898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:09:24.488408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.708763+0000","last_change":"2023-09-12T14:54:58.734734+0000","last_active":"2023-09-12T14:54:59.708763+0000","last_peered":"2023-09-12T14:54:59.708763+0000","last_clean":"2023-09-12T14:54:59.708763+0000","last_became_active":"2023-09-12T14:54:58.730411+0000","last_became_peered":"2023-09-12T14:54:58.730411+0000","last_unstale":"2023-09-12T14:54:59.708763+0000","last_undegraded":"2023-09-12T14:54:59.708763+0000","last_fullsized":"2023-09-12T14:54:59.708763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:04:39.183616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808747+0000","last_change":"2023-09-12T14:54:57.709617+0000","last_active":"2023-09-12T14:54:59.808747+0000","last_peered":"2023-09-12T14:54:59.808747+0000","last_clean":"2023-09-12T14:54:59.808747+0000","last_became_active":"2023-09-12T14:54:57.709171+0000","last_became_peered":"2023-09-12T14:54:57.709171+0000","last_unstale":"2023-09-12T14:54:59.808747+0000","last_undegraded":"2023-09-12T14:54:59.808747+0000","last_fullsized":"2023-09-12T14:54:59.808747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:35:05.562961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806849+0000","last_change":"2023-09-12T14:54:58.722571+0000","last_active":"2023-09-12T14:54:59.806849+0000","last_peered":"2023-09-12T14:54:59.806849+0000","last_clean":"2023-09-12T14:54:59.806849+0000","last_became_active":"2023-09-12T14:54:58.721782+0000","last_became_peered":"2023-09-12T14:54:58.721782+0000","last_unstale":"2023-09-12T14:54:59.806849+0000","last_undegraded":"2023-09-12T14:54:59.806849+0000","last_fullsized":"2023-09-12T14:54:59.806849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:22:58.799616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807614+0000","last_change":"2023-09-12T14:54:57.709417+0000","last_active":"2023-09-12T14:54:59.807614+0000","last_peered":"2023-09-12T14:54:59.807614+0000","last_clean":"2023-09-12T14:54:59.807614+0000","last_became_active":"2023-09-12T14:54:57.708756+0000","last_became_peered":"2023-09-12T14:54:57.708756+0000","last_unstale":"2023-09-12T14:54:59.807614+0000","last_undegraded":"2023-09-12T14:54:59.807614+0000","last_fullsized":"2023-09-12T14:54:59.807614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:50:58.241369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808240+0000","last_change":"2023-09-12T14:54:57.698971+0000","last_active":"2023-09-12T14:54:59.808240+0000","last_peered":"2023-09-12T14:54:59.808240+0000","last_clean":"2023-09-12T14:54:59.808240+0000","last_became_active":"2023-09-12T14:54:57.698574+0000","last_became_peered":"2023-09-12T14:54:57.698574+0000","last_unstale":"2023-09-12T14:54:59.808240+0000","last_undegraded":"2023-09-12T14:54:59.808240+0000","last_fullsized":"2023-09-12T14:54:59.808240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:25:01.213694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808091+0000","last_change":"2023-09-12T14:54:58.748244+0000","last_active":"2023-09-12T14:54:59.808091+0000","last_peered":"2023-09-12T14:54:59.808091+0000","last_clean":"2023-09-12T14:54:59.808091+0000","last_became_active":"2023-09-12T14:54:58.746823+0000","last_became_peered":"2023-09-12T14:54:58.746823+0000","last_unstale":"2023-09-12T14:54:59.808091+0000","last_undegraded":"2023-09-12T14:54:59.808091+0000","last_fullsized":"2023-09-12T14:54:59.808091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:22:58.799616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706293+0000","last_change":"2023-09-12T14:54:57.708138+0000","last_active":"2023-09-12T14:54:59.706293+0000","last_peered":"2023-09-12T14:54:59.706293+0000","last_clean":"2023-09-12T14:54:59.706293+0000","last_became_active":"2023-09-12T14:54:57.705380+0000","last_became_peered":"2023-09-12T14:54:57.705380+0000","last_unstale":"2023-09-12T14:54:59.706293+0000","last_undegraded":"2023-09-12T14:54:59.706293+0000","last_fullsized":"2023-09-12T14:54:59.706293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:34:38.380062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.700719+0000","last_change":"2023-09-12T14:54:58.751072+0000","last_active":"2023-09-12T14:54:59.700719+0000","last_peered":"2023-09-12T14:54:59.700719+0000","last_clean":"2023-09-12T14:54:59.700719+0000","last_became_active":"2023-09-12T14:54:58.750309+0000","last_became_peered":"2023-09-12T14:54:58.750309+0000","last_unstale":"2023-09-12T14:54:59.700719+0000","last_undegraded":"2023-09-12T14:54:59.700719+0000","last_fullsized":"2023-09-12T14:54:59.700719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:58:08.195627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808058+0000","last_change":"2023-09-12T14:54:57.719821+0000","last_active":"2023-09-12T14:54:59.808058+0000","last_peered":"2023-09-12T14:54:59.808058+0000","last_clean":"2023-09-12T14:54:59.808058+0000","last_became_active":"2023-09-12T14:54:57.717672+0000","last_became_peered":"2023-09-12T14:54:57.717672+0000","last_unstale":"2023-09-12T14:54:59.808058+0000","last_undegraded":"2023-09-12T14:54:59.808058+0000","last_fullsized":"2023-09-12T14:54:59.808058+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:25:01.213694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.707290+0000","last_change":"2023-09-12T14:54:58.745459+0000","last_active":"2023-09-12T14:54:59.707290+0000","last_peered":"2023-09-12T14:54:59.707290+0000","last_clean":"2023-09-12T14:54:59.707290+0000","last_became_active":"2023-09-12T14:54:58.744478+0000","last_became_peered":"2023-09-12T14:54:58.744478+0000","last_unstale":"2023-09-12T14:54:59.707290+0000","last_undegraded":"2023-09-12T14:54:59.707290+0000","last_fullsized":"2023-09-12T14:54:59.707290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:07:30.650142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.707357+0000","last_change":"2023-09-12T14:54:57.712325+0000","last_active":"2023-09-12T14:54:59.707357+0000","last_peered":"2023-09-12T14:54:59.707357+0000","last_clean":"2023-09-12T14:54:59.707357+0000","last_became_active":"2023-09-12T14:54:57.711669+0000","last_became_peered":"2023-09-12T14:54:57.711669+0000","last_unstale":"2023-09-12T14:54:59.707357+0000","last_undegraded":"2023-09-12T14:54:59.707357+0000","last_fullsized":"2023-09-12T14:54:59.707357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:25:01.213694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807090+0000","last_change":"2023-09-12T14:54:58.731110+0000","last_active":"2023-09-12T14:54:59.807090+0000","last_peered":"2023-09-12T14:54:59.807090+0000","last_clean":"2023-09-12T14:54:59.807090+0000","last_became_active":"2023-09-12T14:54:58.728351+0000","last_became_peered":"2023-09-12T14:54:58.728351+0000","last_unstale":"2023-09-12T14:54:59.807090+0000","last_undegraded":"2023-09-12T14:54:59.807090+0000","last_fullsized":"2023-09-12T14:54:59.807090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:10:09.339305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.700987+0000","last_change":"2023-09-12T14:54:57.699372+0000","last_active":"2023-09-12T14:54:59.700987+0000","last_peered":"2023-09-12T14:54:59.700987+0000","last_clean":"2023-09-12T14:54:59.700987+0000","last_became_active":"2023-09-12T14:54:57.698970+0000","last_became_peered":"2023-09-12T14:54:57.698970+0000","last_unstale":"2023-09-12T14:54:59.700987+0000","last_undegraded":"2023-09-12T14:54:59.700987+0000","last_fullsized":"2023-09-12T14:54:59.700987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:50:58.241369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.709034+0000","last_change":"2023-09-12T14:54:58.753601+0000","last_active":"2023-09-12T14:54:59.709034+0000","last_peered":"2023-09-12T14:54:59.709034+0000","last_clean":"2023-09-12T14:54:59.709034+0000","last_became_active":"2023-09-12T14:54:58.752414+0000","last_became_peered":"2023-09-12T14:54:58.752414+0000","last_unstale":"2023-09-12T14:54:59.709034+0000","last_undegraded":"2023-09-12T14:54:59.709034+0000","last_fullsized":"2023-09-12T14:54:59.709034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:41:25.126595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807970+0000","last_change":"2023-09-12T14:54:57.699723+0000","last_active":"2023-09-12T14:54:59.807970+0000","last_peered":"2023-09-12T14:54:59.807970+0000","last_clean":"2023-09-12T14:54:59.807970+0000","last_became_active":"2023-09-12T14:54:57.698942+0000","last_became_peered":"2023-09-12T14:54:57.698942+0000","last_unstale":"2023-09-12T14:54:59.807970+0000","last_undegraded":"2023-09-12T14:54:59.807970+0000","last_fullsized":"2023-09-12T14:54:59.807970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:35:05.562961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806782+0000","last_change":"2023-09-12T14:54:58.756683+0000","last_active":"2023-09-12T14:54:59.806782+0000","last_peered":"2023-09-12T14:54:59.806782+0000","last_clean":"2023-09-12T14:54:59.806782+0000","last_became_active":"2023-09-12T14:54:58.755952+0000","last_became_peered":"2023-09-12T14:54:58.755952+0000","last_unstale":"2023-09-12T14:54:59.806782+0000","last_undegraded":"2023-09-12T14:54:59.806782+0000","last_fullsized":"2023-09-12T14:54:59.806782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:07:26.691746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806819+0000","last_change":"2023-09-12T14:54:57.708009+0000","last_active":"2023-09-12T14:54:59.806819+0000","last_peered":"2023-09-12T14:54:59.806819+0000","last_clean":"2023-09-12T14:54:59.806819+0000","last_became_active":"2023-09-12T14:54:57.707617+0000","last_became_peered":"2023-09-12T14:54:57.707617+0000","last_unstale":"2023-09-12T14:54:59.806819+0000","last_undegraded":"2023-09-12T14:54:59.806819+0000","last_fullsized":"2023-09-12T14:54:59.806819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:10:06.188199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806973+0000","last_change":"2023-09-12T14:54:58.747555+0000","last_active":"2023-09-12T14:54:59.806973+0000","last_peered":"2023-09-12T14:54:59.806973+0000","last_clean":"2023-09-12T14:54:59.806973+0000","last_became_active":"2023-09-12T14:54:58.745580+0000","last_became_peered":"2023-09-12T14:54:58.745580+0000","last_unstale":"2023-09-12T14:54:59.806973+0000","last_undegraded":"2023-09-12T14:54:59.806973+0000","last_fullsized":"2023-09-12T14:54:59.806973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:33.647832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706246+0000","last_change":"2023-09-12T14:54:57.713557+0000","last_active":"2023-09-12T14:54:59.706246+0000","last_peered":"2023-09-12T14:54:59.706246+0000","last_clean":"2023-09-12T14:54:59.706246+0000","last_became_active":"2023-09-12T14:54:57.712038+0000","last_became_peered":"2023-09-12T14:54:57.712038+0000","last_unstale":"2023-09-12T14:54:59.706246+0000","last_undegraded":"2023-09-12T14:54:59.706246+0000","last_fullsized":"2023-09-12T14:54:59.706246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:22:57.788678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806857+0000","last_change":"2023-09-12T14:54:58.754016+0000","last_active":"2023-09-12T14:54:59.806857+0000","last_peered":"2023-09-12T14:54:59.806857+0000","last_clean":"2023-09-12T14:54:59.806857+0000","last_became_active":"2023-09-12T14:54:58.753380+0000","last_became_peered":"2023-09-12T14:54:58.753380+0000","last_unstale":"2023-09-12T14:54:59.806857+0000","last_undegraded":"2023-09-12T14:54:59.806857+0000","last_fullsized":"2023-09-12T14:54:59.806857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:16:18.487363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805823+0000","last_change":"2023-09-12T14:54:57.713543+0000","last_active":"2023-09-12T14:54:59.805823+0000","last_peered":"2023-09-12T14:54:59.805823+0000","last_clean":"2023-09-12T14:54:59.805823+0000","last_became_active":"2023-09-12T14:54:57.712819+0000","last_became_peered":"2023-09-12T14:54:57.712819+0000","last_unstale":"2023-09-12T14:54:59.805823+0000","last_undegraded":"2023-09-12T14:54:59.805823+0000","last_fullsized":"2023-09-12T14:54:59.805823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:10:06.188199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805969+0000","last_change":"2023-09-12T14:54:58.745224+0000","last_active":"2023-09-12T14:54:59.805969+0000","last_peered":"2023-09-12T14:54:59.805969+0000","last_clean":"2023-09-12T14:54:59.805969+0000","last_became_active":"2023-09-12T14:54:58.744566+0000","last_became_peered":"2023-09-12T14:54:58.744566+0000","last_unstale":"2023-09-12T14:54:59.805969+0000","last_undegraded":"2023-09-12T14:54:59.805969+0000","last_fullsized":"2023-09-12T14:54:59.805969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:56:13.426275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805117+0000","last_change":"2023-09-12T14:54:57.683795+0000","last_active":"2023-09-12T14:54:59.805117+0000","last_peered":"2023-09-12T14:54:59.805117+0000","last_clean":"2023-09-12T14:54:59.805117+0000","last_became_active":"2023-09-12T14:54:57.683309+0000","last_became_peered":"2023-09-12T14:54:57.683309+0000","last_unstale":"2023-09-12T14:54:59.805117+0000","last_undegraded":"2023-09-12T14:54:59.805117+0000","last_fullsized":"2023-09-12T14:54:59.805117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:38:46.256403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806420+0000","last_change":"2023-09-12T14:54:58.744197+0000","last_active":"2023-09-12T14:54:59.806420+0000","last_peered":"2023-09-12T14:54:59.806420+0000","last_clean":"2023-09-12T14:54:59.806420+0000","last_became_active":"2023-09-12T14:54:58.743668+0000","last_became_peered":"2023-09-12T14:54:58.743668+0000","last_unstale":"2023-09-12T14:54:59.806420+0000","last_undegraded":"2023-09-12T14:54:59.806420+0000","last_fullsized":"2023-09-12T14:54:59.806420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:33:30.961102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.709354+0000","last_change":"2023-09-12T14:54:57.704349+0000","last_active":"2023-09-12T14:54:59.709354+0000","last_peered":"2023-09-12T14:54:59.709354+0000","last_clean":"2023-09-12T14:54:59.709354+0000","last_became_active":"2023-09-12T14:54:57.703134+0000","last_became_peered":"2023-09-12T14:54:57.703134+0000","last_unstale":"2023-09-12T14:54:59.709354+0000","last_undegraded":"2023-09-12T14:54:59.709354+0000","last_fullsized":"2023-09-12T14:54:59.709354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:25:54.071276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807199+0000","last_change":"2023-09-12T14:54:58.752753+0000","last_active":"2023-09-12T14:54:59.807199+0000","last_peered":"2023-09-12T14:54:59.807199+0000","last_clean":"2023-09-12T14:54:59.807199+0000","last_became_active":"2023-09-12T14:54:58.751852+0000","last_became_peered":"2023-09-12T14:54:58.751852+0000","last_unstale":"2023-09-12T14:54:59.807199+0000","last_undegraded":"2023-09-12T14:54:59.807199+0000","last_fullsized":"2023-09-12T14:54:59.807199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:23:41.038660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806233+0000","last_change":"2023-09-12T14:54:57.691156+0000","last_active":"2023-09-12T14:54:59.806233+0000","last_peered":"2023-09-12T14:54:59.806233+0000","last_clean":"2023-09-12T14:54:59.806233+0000","last_became_active":"2023-09-12T14:54:57.690769+0000","last_became_peered":"2023-09-12T14:54:57.690769+0000","last_unstale":"2023-09-12T14:54:59.806233+0000","last_undegraded":"2023-09-12T14:54:59.806233+0000","last_fullsized":"2023-09-12T14:54:59.806233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:48:28.111452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807364+0000","last_change":"2023-09-12T14:54:58.755613+0000","last_active":"2023-09-12T14:54:59.807364+0000","last_peered":"2023-09-12T14:54:59.807364+0000","last_clean":"2023-09-12T14:54:59.807364+0000","last_became_active":"2023-09-12T14:54:58.755005+0000","last_became_peered":"2023-09-12T14:54:58.755005+0000","last_unstale":"2023-09-12T14:54:59.807364+0000","last_undegraded":"2023-09-12T14:54:59.807364+0000","last_fullsized":"2023-09-12T14:54:59.807364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:27:30.157653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.701127+0000","last_change":"2023-09-12T14:54:57.686468+0000","last_active":"2023-09-12T14:54:59.701127+0000","last_peered":"2023-09-12T14:54:59.701127+0000","last_clean":"2023-09-12T14:54:59.701127+0000","last_became_active":"2023-09-12T14:54:57.686059+0000","last_became_peered":"2023-09-12T14:54:57.686059+0000","last_unstale":"2023-09-12T14:54:59.701127+0000","last_undegraded":"2023-09-12T14:54:59.701127+0000","last_fullsized":"2023-09-12T14:54:59.701127+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:35:05.562961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706190+0000","last_change":"2023-09-12T14:54:58.751002+0000","last_active":"2023-09-12T14:54:59.706190+0000","last_peered":"2023-09-12T14:54:59.706190+0000","last_clean":"2023-09-12T14:54:59.706190+0000","last_became_active":"2023-09-12T14:54:58.750380+0000","last_became_peered":"2023-09-12T14:54:58.750380+0000","last_unstale":"2023-09-12T14:54:59.706190+0000","last_undegraded":"2023-09-12T14:54:59.706190+0000","last_fullsized":"2023-09-12T14:54:59.706190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:56:32.731297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805597+0000","last_change":"2023-09-12T14:54:57.680566+0000","last_active":"2023-09-12T14:54:59.805597+0000","last_peered":"2023-09-12T14:54:59.805597+0000","last_clean":"2023-09-12T14:54:59.805597+0000","last_became_active":"2023-09-12T14:54:57.680113+0000","last_became_peered":"2023-09-12T14:54:57.680113+0000","last_unstale":"2023-09-12T14:54:59.805597+0000","last_undegraded":"2023-09-12T14:54:59.805597+0000","last_fullsized":"2023-09-12T14:54:59.805597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:48:51.271671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806912+0000","last_change":"2023-09-12T14:54:58.746352+0000","last_active":"2023-09-12T14:54:59.806912+0000","last_peered":"2023-09-12T14:54:59.806912+0000","last_clean":"2023-09-12T14:54:59.806912+0000","last_became_active":"2023-09-12T14:54:58.745469+0000","last_became_peered":"2023-09-12T14:54:58.745469+0000","last_unstale":"2023-09-12T14:54:59.806912+0000","last_undegraded":"2023-09-12T14:54:59.806912+0000","last_fullsized":"2023-09-12T14:54:59.806912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:56:32.731297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-12T14:54:58.722614+0000","last_change":"2023-09-12T14:54:46.965446+0000","last_active":"2023-09-12T14:54:58.722614+0000","last_peered":"2023-09-12T14:54:58.722614+0000","last_clean":"2023-09-12T14:54:58.722614+0000","last_became_active":"2023-09-12T14:54:46.963236+0000","last_became_peered":"2023-09-12T14:54:46.963236+0000","last_unstale":"2023-09-12T14:54:58.722614+0000","last_undegraded":"2023-09-12T14:54:58.722614+0000","last_fullsized":"2023-09-12T14:54:58.722614+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:45.804232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:45.804232+0000","last_clean_scrub_stamp":"2023-09-12T14:54:45.804232+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:24:45.670513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.705459+0000","last_change":"2023-09-12T14:54:57.707645+0000","last_active":"2023-09-12T14:54:59.705459+0000","last_peered":"2023-09-12T14:54:59.705459+0000","last_clean":"2023-09-12T14:54:59.705459+0000","last_became_active":"2023-09-12T14:54:57.705665+0000","last_became_peered":"2023-09-12T14:54:57.705665+0000","last_unstale":"2023-09-12T14:54:59.705459+0000","last_undegraded":"2023-09-12T14:54:59.705459+0000","last_fullsized":"2023-09-12T14:54:59.705459+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:48:28.111452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808625+0000","last_change":"2023-09-12T14:54:58.745179+0000","last_active":"2023-09-12T14:54:59.808625+0000","last_peered":"2023-09-12T14:54:59.808625+0000","last_clean":"2023-09-12T14:54:59.808625+0000","last_became_active":"2023-09-12T14:54:58.744548+0000","last_became_peered":"2023-09-12T14:54:58.744548+0000","last_unstale":"2023-09-12T14:54:59.808625+0000","last_undegraded":"2023-09-12T14:54:59.808625+0000","last_fullsized":"2023-09-12T14:54:59.808625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:10:09.339305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806036+0000","last_change":"2023-09-12T14:54:57.719515+0000","last_active":"2023-09-12T14:54:59.806036+0000","last_peered":"2023-09-12T14:54:59.806036+0000","last_clean":"2023-09-12T14:54:59.806036+0000","last_became_active":"2023-09-12T14:54:57.717582+0000","last_became_peered":"2023-09-12T14:54:57.717582+0000","last_unstale":"2023-09-12T14:54:59.806036+0000","last_undegraded":"2023-09-12T14:54:59.806036+0000","last_fullsized":"2023-09-12T14:54:59.806036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:48:51.271671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806471+0000","last_change":"2023-09-12T14:54:58.715137+0000","last_active":"2023-09-12T14:54:59.806471+0000","last_peered":"2023-09-12T14:54:59.806471+0000","last_clean":"2023-09-12T14:54:59.806471+0000","last_became_active":"2023-09-12T14:54:58.713883+0000","last_became_peered":"2023-09-12T14:54:58.713883+0000","last_unstale":"2023-09-12T14:54:59.806471+0000","last_undegraded":"2023-09-12T14:54:59.806471+0000","last_fullsized":"2023-09-12T14:54:59.806471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:20:59.118588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805239+0000","last_change":"2023-09-12T14:54:57.691863+0000","last_active":"2023-09-12T14:54:59.805239+0000","last_peered":"2023-09-12T14:54:59.805239+0000","last_clean":"2023-09-12T14:54:59.805239+0000","last_became_active":"2023-09-12T14:54:57.691453+0000","last_became_peered":"2023-09-12T14:54:57.691453+0000","last_unstale":"2023-09-12T14:54:59.805239+0000","last_undegraded":"2023-09-12T14:54:59.805239+0000","last_fullsized":"2023-09-12T14:54:59.805239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:01:34.927854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.700716+0000","last_change":"2023-09-12T14:54:58.732432+0000","last_active":"2023-09-12T14:54:59.700716+0000","last_peered":"2023-09-12T14:54:59.700716+0000","last_clean":"2023-09-12T14:54:59.700716+0000","last_became_active":"2023-09-12T14:54:58.731662+0000","last_became_peered":"2023-09-12T14:54:58.731662+0000","last_unstale":"2023-09-12T14:54:59.700716+0000","last_undegraded":"2023-09-12T14:54:59.700716+0000","last_fullsized":"2023-09-12T14:54:59.700716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:09:24.488408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706495+0000","last_change":"2023-09-12T14:54:57.714059+0000","last_active":"2023-09-12T14:54:59.706495+0000","last_peered":"2023-09-12T14:54:59.706495+0000","last_clean":"2023-09-12T14:54:59.706495+0000","last_became_active":"2023-09-12T14:54:57.712854+0000","last_became_peered":"2023-09-12T14:54:57.712854+0000","last_unstale":"2023-09-12T14:54:59.706495+0000","last_undegraded":"2023-09-12T14:54:59.706495+0000","last_fullsized":"2023-09-12T14:54:59.706495+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:56:12.415337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.709032+0000","last_change":"2023-09-12T14:54:58.713195+0000","last_active":"2023-09-12T14:54:59.709032+0000","last_peered":"2023-09-12T14:54:59.709032+0000","last_clean":"2023-09-12T14:54:59.709032+0000","last_became_active":"2023-09-12T14:54:58.712392+0000","last_became_peered":"2023-09-12T14:54:58.712392+0000","last_unstale":"2023-09-12T14:54:59.709032+0000","last_undegraded":"2023-09-12T14:54:59.709032+0000","last_fullsized":"2023-09-12T14:54:59.709032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:01:35.938792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.701021+0000","last_change":"2023-09-12T14:54:57.691816+0000","last_active":"2023-09-12T14:54:59.701021+0000","last_peered":"2023-09-12T14:54:59.701021+0000","last_clean":"2023-09-12T14:54:59.701021+0000","last_became_active":"2023-09-12T14:54:57.691410+0000","last_became_peered":"2023-09-12T14:54:57.691410+0000","last_unstale":"2023-09-12T14:54:59.701021+0000","last_undegraded":"2023-09-12T14:54:59.701021+0000","last_fullsized":"2023-09-12T14:54:59.701021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:01:34.927854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706911+0000","last_change":"2023-09-12T14:54:58.748697+0000","last_active":"2023-09-12T14:54:59.706911+0000","last_peered":"2023-09-12T14:54:59.706911+0000","last_clean":"2023-09-12T14:54:59.706911+0000","last_became_active":"2023-09-12T14:54:58.748072+0000","last_became_peered":"2023-09-12T14:54:58.748072+0000","last_unstale":"2023-09-12T14:54:59.706911+0000","last_undegraded":"2023-09-12T14:54:59.706911+0000","last_fullsized":"2023-09-12T14:54:59.706911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:27:30.157653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807241+0000","last_change":"2023-09-12T14:54:57.718970+0000","last_active":"2023-09-12T14:54:59.807241+0000","last_peered":"2023-09-12T14:54:59.807241+0000","last_clean":"2023-09-12T14:54:59.807241+0000","last_became_active":"2023-09-12T14:54:57.717410+0000","last_became_peered":"2023-09-12T14:54:57.717410+0000","last_unstale":"2023-09-12T14:54:59.807241+0000","last_undegraded":"2023-09-12T14:54:59.807241+0000","last_fullsized":"2023-09-12T14:54:59.807241+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:22:57.788678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.701025+0000","last_change":"2023-09-12T14:54:58.733797+0000","last_active":"2023-09-12T14:54:59.701025+0000","last_peered":"2023-09-12T14:54:59.701025+0000","last_clean":"2023-09-12T14:54:59.701025+0000","last_became_active":"2023-09-12T14:54:58.732869+0000","last_became_peered":"2023-09-12T14:54:58.732869+0000","last_unstale":"2023-09-12T14:54:59.701025+0000","last_undegraded":"2023-09-12T14:54:59.701025+0000","last_fullsized":"2023-09-12T14:54:59.701025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:22:58.799616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.701305+0000","last_change":"2023-09-12T14:54:57.703103+0000","last_active":"2023-09-12T14:54:59.701305+0000","last_peered":"2023-09-12T14:54:59.701305+0000","last_clean":"2023-09-12T14:54:59.701305+0000","last_became_active":"2023-09-12T14:54:57.702706+0000","last_became_peered":"2023-09-12T14:54:57.702706+0000","last_unstale":"2023-09-12T14:54:59.701305+0000","last_undegraded":"2023-09-12T14:54:59.701305+0000","last_fullsized":"2023-09-12T14:54:59.701305+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:34:38.380062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.809481+0000","last_change":"2023-09-12T14:54:58.745944+0000","last_active":"2023-09-12T14:54:59.809481+0000","last_peered":"2023-09-12T14:54:59.809481+0000","last_clean":"2023-09-12T14:54:59.809481+0000","last_became_active":"2023-09-12T14:54:58.744677+0000","last_became_peered":"2023-09-12T14:54:58.744677+0000","last_unstale":"2023-09-12T14:54:59.809481+0000","last_undegraded":"2023-09-12T14:54:59.809481+0000","last_fullsized":"2023-09-12T14:54:59.809481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:26:45.799536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808147+0000","last_change":"2023-09-12T14:54:57.719402+0000","last_active":"2023-09-12T14:54:59.808147+0000","last_peered":"2023-09-12T14:54:59.808147+0000","last_clean":"2023-09-12T14:54:59.808147+0000","last_became_active":"2023-09-12T14:54:57.718815+0000","last_became_peered":"2023-09-12T14:54:57.718815+0000","last_unstale":"2023-09-12T14:54:59.808147+0000","last_undegraded":"2023-09-12T14:54:59.808147+0000","last_fullsized":"2023-09-12T14:54:59.808147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:20:58.107650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807282+0000","last_change":"2023-09-12T14:54:58.732619+0000","last_active":"2023-09-12T14:54:59.807282+0000","last_peered":"2023-09-12T14:54:59.807282+0000","last_clean":"2023-09-12T14:54:59.807282+0000","last_became_active":"2023-09-12T14:54:58.731845+0000","last_became_peered":"2023-09-12T14:54:58.731845+0000","last_unstale":"2023-09-12T14:54:59.807282+0000","last_undegraded":"2023-09-12T14:54:59.807282+0000","last_fullsized":"2023-09-12T14:54:59.807282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:33:30.961102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805781+0000","last_change":"2023-09-12T14:54:57.717947+0000","last_active":"2023-09-12T14:54:59.805781+0000","last_peered":"2023-09-12T14:54:59.805781+0000","last_clean":"2023-09-12T14:54:59.805781+0000","last_became_active":"2023-09-12T14:54:57.716465+0000","last_became_peered":"2023-09-12T14:54:57.716465+0000","last_unstale":"2023-09-12T14:54:59.805781+0000","last_undegraded":"2023-09-12T14:54:59.805781+0000","last_fullsized":"2023-09-12T14:54:59.805781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:01:34.927854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.708729+0000","last_change":"2023-09-12T14:54:58.734846+0000","last_active":"2023-09-12T14:54:59.708729+0000","last_peered":"2023-09-12T14:54:59.708729+0000","last_clean":"2023-09-12T14:54:59.708729+0000","last_became_active":"2023-09-12T14:54:58.733232+0000","last_became_peered":"2023-09-12T14:54:58.733232+0000","last_unstale":"2023-09-12T14:54:59.708729+0000","last_undegraded":"2023-09-12T14:54:59.708729+0000","last_fullsized":"2023-09-12T14:54:59.708729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:48:52.282609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.700759+0000","last_change":"2023-09-12T14:54:57.693075+0000","last_active":"2023-09-12T14:54:59.700759+0000","last_peered":"2023-09-12T14:54:59.700759+0000","last_clean":"2023-09-12T14:54:59.700759+0000","last_became_active":"2023-09-12T14:54:57.692190+0000","last_became_peered":"2023-09-12T14:54:57.692190+0000","last_unstale":"2023-09-12T14:54:59.700759+0000","last_undegraded":"2023-09-12T14:54:59.700759+0000","last_fullsized":"2023-09-12T14:54:59.700759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:41:24.115657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808265+0000","last_change":"2023-09-12T14:54:58.748674+0000","last_active":"2023-09-12T14:54:59.808265+0000","last_peered":"2023-09-12T14:54:59.808265+0000","last_clean":"2023-09-12T14:54:59.808265+0000","last_became_active":"2023-09-12T14:54:58.747570+0000","last_became_peered":"2023-09-12T14:54:58.747570+0000","last_unstale":"2023-09-12T14:54:59.808265+0000","last_undegraded":"2023-09-12T14:54:59.808265+0000","last_fullsized":"2023-09-12T14:54:59.808265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:56:13.426275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.700971+0000","last_change":"2023-09-12T14:54:57.690103+0000","last_active":"2023-09-12T14:54:59.700971+0000","last_peered":"2023-09-12T14:54:59.700971+0000","last_clean":"2023-09-12T14:54:59.700971+0000","last_became_active":"2023-09-12T14:54:57.688945+0000","last_became_peered":"2023-09-12T14:54:57.688945+0000","last_unstale":"2023-09-12T14:54:59.700971+0000","last_undegraded":"2023-09-12T14:54:59.700971+0000","last_fullsized":"2023-09-12T14:54:59.700971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:25:01.213694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808786+0000","last_change":"2023-09-12T14:54:58.744905+0000","last_active":"2023-09-12T14:54:59.808786+0000","last_peered":"2023-09-12T14:54:59.808786+0000","last_clean":"2023-09-12T14:54:59.808786+0000","last_became_active":"2023-09-12T14:54:58.744132+0000","last_became_peered":"2023-09-12T14:54:58.744132+0000","last_unstale":"2023-09-12T14:54:59.808786+0000","last_undegraded":"2023-09-12T14:54:59.808786+0000","last_fullsized":"2023-09-12T14:54:59.808786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:33:30.961102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807246+0000","last_change":"2023-09-12T14:54:57.718508+0000","last_active":"2023-09-12T14:54:59.807246+0000","last_peered":"2023-09-12T14:54:59.807246+0000","last_clean":"2023-09-12T14:54:59.807246+0000","last_became_active":"2023-09-12T14:54:57.716751+0000","last_became_peered":"2023-09-12T14:54:57.716751+0000","last_unstale":"2023-09-12T14:54:59.807246+0000","last_undegraded":"2023-09-12T14:54:59.807246+0000","last_fullsized":"2023-09-12T14:54:59.807246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:34:38.380062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805940+0000","last_change":"2023-09-12T14:54:58.747254+0000","last_active":"2023-09-12T14:54:59.805940+0000","last_peered":"2023-09-12T14:54:59.805940+0000","last_clean":"2023-09-12T14:54:59.805940+0000","last_became_active":"2023-09-12T14:54:58.746726+0000","last_became_peered":"2023-09-12T14:54:58.746726+0000","last_unstale":"2023-09-12T14:54:59.805940+0000","last_undegraded":"2023-09-12T14:54:59.805940+0000","last_fullsized":"2023-09-12T14:54:59.805940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:22:58.799616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706879+0000","last_change":"2023-09-12T14:54:58.720140+0000","last_active":"2023-09-12T14:54:59.706879+0000","last_peered":"2023-09-12T14:54:59.706879+0000","last_clean":"2023-09-12T14:54:59.706879+0000","last_became_active":"2023-09-12T14:54:58.717751+0000","last_became_peered":"2023-09-12T14:54:58.717751+0000","last_unstale":"2023-09-12T14:54:59.706879+0000","last_undegraded":"2023-09-12T14:54:59.706879+0000","last_fullsized":"2023-09-12T14:54:59.706879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:13:39.293184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806375+0000","last_change":"2023-09-12T14:54:57.717851+0000","last_active":"2023-09-12T14:54:59.806375+0000","last_peered":"2023-09-12T14:54:59.806375+0000","last_clean":"2023-09-12T14:54:59.806375+0000","last_became_active":"2023-09-12T14:54:57.716821+0000","last_became_peered":"2023-09-12T14:54:57.716821+0000","last_unstale":"2023-09-12T14:54:59.806375+0000","last_undegraded":"2023-09-12T14:54:59.806375+0000","last_fullsized":"2023-09-12T14:54:59.806375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:34.954926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.705907+0000","last_change":"2023-09-12T14:54:58.722319+0000","last_active":"2023-09-12T14:54:59.705907+0000","last_peered":"2023-09-12T14:54:59.705907+0000","last_clean":"2023-09-12T14:54:59.705907+0000","last_became_active":"2023-09-12T14:54:58.719871+0000","last_became_peered":"2023-09-12T14:54:58.719871+0000","last_unstale":"2023-09-12T14:54:59.705907+0000","last_undegraded":"2023-09-12T14:54:59.705907+0000","last_fullsized":"2023-09-12T14:54:59.705907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:07:26.691746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807030+0000","last_change":"2023-09-12T14:54:57.718179+0000","last_active":"2023-09-12T14:54:59.807030+0000","last_peered":"2023-09-12T14:54:59.807030+0000","last_clean":"2023-09-12T14:54:59.807030+0000","last_became_active":"2023-09-12T14:54:57.716712+0000","last_became_peered":"2023-09-12T14:54:57.716712+0000","last_unstale":"2023-09-12T14:54:59.807030+0000","last_undegraded":"2023-09-12T14:54:59.807030+0000","last_fullsized":"2023-09-12T14:54:59.807030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:25:01.213694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806750+0000","last_change":"2023-09-12T14:54:58.728727+0000","last_active":"2023-09-12T14:54:59.806750+0000","last_peered":"2023-09-12T14:54:59.806750+0000","last_clean":"2023-09-12T14:54:59.806750+0000","last_became_active":"2023-09-12T14:54:58.727977+0000","last_became_peered":"2023-09-12T14:54:58.727977+0000","last_unstale":"2023-09-12T14:54:59.806750+0000","last_undegraded":"2023-09-12T14:54:59.806750+0000","last_fullsized":"2023-09-12T14:54:59.806750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:09:24.488408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.809447+0000","last_change":"2023-09-12T14:54:57.683980+0000","last_active":"2023-09-12T14:54:59.809447+0000","last_peered":"2023-09-12T14:54:59.809447+0000","last_clean":"2023-09-12T14:54:59.809447+0000","last_became_active":"2023-09-12T14:54:57.683572+0000","last_became_peered":"2023-09-12T14:54:57.683572+0000","last_unstale":"2023-09-12T14:54:59.809447+0000","last_undegraded":"2023-09-12T14:54:59.809447+0000","last_fullsized":"2023-09-12T14:54:59.809447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:10:06.188199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.809386+0000","last_change":"2023-09-12T14:54:58.758048+0000","last_active":"2023-09-12T14:54:59.809386+0000","last_peered":"2023-09-12T14:54:59.809386+0000","last_clean":"2023-09-12T14:54:59.809386+0000","last_became_active":"2023-09-12T14:54:58.757449+0000","last_became_peered":"2023-09-12T14:54:58.757449+0000","last_unstale":"2023-09-12T14:54:59.809386+0000","last_undegraded":"2023-09-12T14:54:59.809386+0000","last_fullsized":"2023-09-12T14:54:59.809386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:07:26.691746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807331+0000","last_change":"2023-09-12T14:54:57.709447+0000","last_active":"2023-09-12T14:54:59.807331+0000","last_peered":"2023-09-12T14:54:59.807331+0000","last_clean":"2023-09-12T14:54:59.807331+0000","last_became_active":"2023-09-12T14:54:57.709126+0000","last_became_peered":"2023-09-12T14:54:57.709126+0000","last_unstale":"2023-09-12T14:54:59.807331+0000","last_undegraded":"2023-09-12T14:54:59.807331+0000","last_fullsized":"2023-09-12T14:54:59.807331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:34:38.380062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.337}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":70988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":806912,"data_stored":660885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-12T14:55:01.994 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-12T14:55:02.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f58803d5640 1 -- 172.21.15.131:0/3670664003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5864003980 msgr2=0x7f5864003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:02.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f58803d5640 1 --2- 172.21.15.131:0/3670664003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5864003980 0x7f5864003d80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5868006fd0 tx=0x7f586801cf70 comp rx=0 tx=0).stop 2023-09-12T14:55:02.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f58803d5640 1 -- 172.21.15.131:0/3670664003 shutdown_connections 2023-09-12T14:55:02.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f58803d5640 1 --2- 172.21.15.131:0/3670664003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5864003980 0x7f5864003d80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:02.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f58803d5640 1 -- 172.21.15.131:0/3670664003 >> 172.21.15.131:0/3670664003 conn(0x7f5878069840 msgr2=0x7f58780cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:02.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f58803d5640 1 -- 172.21.15.131:0/3670664003 shutdown_connections 2023-09-12T14:55:02.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f58803d5640 1 -- 172.21.15.131:0/3670664003 wait complete. 2023-09-12T14:55:02.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f58803d5640 1 Processor -- start 2023-09-12T14:55:02.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f58803d5640 1 -- start start 2023-09-12T14:55:02.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f58803d5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5864003980 0x7f5878146650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:02.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f58803d5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58780b0be0 0x7f58780b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:02.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f58803d5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58780b1a90 0x7f587815b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:02.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f58803d5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5878146b90 con 0x7f5864003980 2023-09-12T14:55:02.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f58803d5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58780b1fa0 con 0x7f58780b0be0 2023-09-12T14:55:02.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f58803d5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58780b2280 con 0x7f58780b1a90 2023-09-12T14:55:02.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f587e171640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5864003980 0x7f5878146650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:02.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f587e171640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5864003980 0x7f5878146650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38774/0 (socket says 172.21.15.131:38774) 2023-09-12T14:55:02.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f587e171640 1 -- 172.21.15.131:0/263911746 learned_addr learned my addr 172.21.15.131:0/263911746 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:02.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f587d970640 1 --2- 172.21.15.131:0/263911746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58780b0be0 0x7f58780b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:02.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f587e171640 1 -- 172.21.15.131:0/263911746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58780b1a90 msgr2=0x7f587815b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f587e972640 1 --2- 172.21.15.131:0/263911746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58780b1a90 0x7f587815b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f587e171640 1 --2- 172.21.15.131:0/263911746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58780b1a90 0x7f587815b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f587e171640 1 -- 172.21.15.131:0/263911746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58780b0be0 msgr2=0x7f58780b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f587e171640 1 --2- 172.21.15.131:0/263911746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58780b0be0 0x7f58780b1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f587e171640 1 -- 172.21.15.131:0/263911746 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58780b0100 con 0x7f5864003980 2023-09-12T14:55:02.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f587e171640 1 --2- 172.21.15.131:0/263911746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5864003980 0x7f5878146650 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5868001940 tx=0x7f5868001d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:02.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f586effd640 1 -- 172.21.15.131:0/263911746 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58680b3070 con 0x7f5864003980 2023-09-12T14:55:02.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f586effd640 1 -- 172.21.15.131:0/263911746 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5868006320 con 0x7f5864003980 2023-09-12T14:55:02.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f58803d5640 1 -- 172.21.15.131:0/263911746 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58780b03e0 con 0x7f5864003980 2023-09-12T14:55:02.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f586effd640 1 -- 172.21.15.131:0/263911746 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5868003600 con 0x7f5864003980 2023-09-12T14:55:02.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.149+0000 7f58803d5640 1 -- 172.21.15.131:0/263911746 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58780b1060 con 0x7f5864003980 2023-09-12T14:55:02.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.153+0000 7f58803d5640 1 -- 172.21.15.131:0/263911746 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5878073580 con 0x7f5864003980 2023-09-12T14:55:02.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.153+0000 7f586effd640 1 -- 172.21.15.131:0/263911746 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f58680ab020 con 0x7f5864003980 2023-09-12T14:55:02.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.153+0000 7f586effd640 1 --2- 172.21.15.131:0/263911746 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f585c065b50 0x7f585c068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:02.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.153+0000 7f587d970640 1 --2- 172.21.15.131:0/263911746 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f585c065b50 0x7f585c068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:02.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.153+0000 7f586effd640 1 -- 172.21.15.131:0/263911746 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f586811d410 con 0x7f5864003980 2023-09-12T14:55:02.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.153+0000 7f587d970640 1 --2- 172.21.15.131:0/263911746 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f585c065b50 0x7f585c068010 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5874005fd0 tx=0x7f5874005f40 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:02.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.157+0000 7f586effd640 1 -- 172.21.15.131:0/263911746 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f58680eadf0 con 0x7f5864003980 2023-09-12T14:55:02.218 INFO:tasks.ceph.mds.d.smithi187.stderr:2023-09-12T14:55:02.214+0000 7f4a64e4e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-12T14:55:02.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.305+0000 7f58803d5640 1 -- 172.21.15.131:0/263911746 --> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f587806fba0 con 0x7f585c065b50 2023-09-12T14:55:02.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.325+0000 7f586effd640 1 -- 172.21.15.131:0/263911746 <== mgr.4200 v2:172.21.15.131:6832/2298103651 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264611 (secure 0 0 0) 0x7f5874016070 con 0x7f585c065b50 2023-09-12T14:55:02.326 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:02.329 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-09-12T14:55:02.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.329+0000 7f58803d5640 1 -- 172.21.15.131:0/263911746 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f585c065b50 msgr2=0x7f585c068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:02.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.329+0000 7f58803d5640 1 --2- 172.21.15.131:0/263911746 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f585c065b50 0x7f585c068010 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5874005fd0 tx=0x7f5874005f40 comp rx=0 tx=0).stop 2023-09-12T14:55:02.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.329+0000 7f58803d5640 1 -- 172.21.15.131:0/263911746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5864003980 msgr2=0x7f5878146650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:02.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.329+0000 7f58803d5640 1 --2- 172.21.15.131:0/263911746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5864003980 0x7f5878146650 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5868001940 tx=0x7f5868001d50 comp rx=0 tx=0).stop 2023-09-12T14:55:02.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.329+0000 7f58803d5640 1 -- 172.21.15.131:0/263911746 shutdown_connections 2023-09-12T14:55:02.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.329+0000 7f58803d5640 1 --2- 172.21.15.131:0/263911746 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f585c065b50 0x7f585c068010 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:02.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.329+0000 7f58803d5640 1 --2- 172.21.15.131:0/263911746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58780b1a90 0x7f587815b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:02.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.329+0000 7f58803d5640 1 --2- 172.21.15.131:0/263911746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58780b0be0 0x7f58780b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:02.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.329+0000 7f58803d5640 1 --2- 172.21.15.131:0/263911746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5864003980 0x7f5878146650 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:02.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.329+0000 7f58803d5640 1 -- 172.21.15.131:0/263911746 >> 172.21.15.131:0/263911746 conn(0x7f5878069840 msgr2=0x7f587806bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:02.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.329+0000 7f58803d5640 1 -- 172.21.15.131:0/263911746 shutdown_connections 2023-09-12T14:55:02.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.329+0000 7f58803d5640 1 -- 172.21.15.131:0/263911746 wait complete. 2023-09-12T14:55:02.346 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-09-12T14:55:01.580348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":144,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218980,"kb_used_data":2796,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755740,"statfs":{"total":773094113280,"available":772869877760,"internally_reserved":0,"allocated":2863104,"data_stored":1739121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":63,"apply_latency_ms":63,"commit_latency_ns":63000000,"apply_latency_ns":63000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.923057"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806828+0000","last_change":"2023-09-12T14:54:58.747316+0000","last_active":"2023-09-12T14:54:59.806828+0000","last_peered":"2023-09-12T14:54:59.806828+0000","last_clean":"2023-09-12T14:54:59.806828+0000","last_became_active":"2023-09-12T14:54:58.746402+0000","last_became_peered":"2023-09-12T14:54:58.746402+0000","last_unstale":"2023-09-12T14:54:59.806828+0000","last_undegraded":"2023-09-12T14:54:59.806828+0000","last_fullsized":"2023-09-12T14:54:59.806828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:26:45.799536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.701256+0000","last_change":"2023-09-12T14:54:58.726405+0000","last_active":"2023-09-12T14:54:59.701256+0000","last_peered":"2023-09-12T14:54:59.701256+0000","last_clean":"2023-09-12T14:54:59.701256+0000","last_became_active":"2023-09-12T14:54:58.723598+0000","last_became_peered":"2023-09-12T14:54:58.723598+0000","last_unstale":"2023-09-12T14:54:59.701256+0000","last_undegraded":"2023-09-12T14:54:59.701256+0000","last_fullsized":"2023-09-12T14:54:59.701256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:56:13.426275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806975+0000","last_change":"2023-09-12T14:54:58.735800+0000","last_active":"2023-09-12T14:54:59.806975+0000","last_peered":"2023-09-12T14:54:59.806975+0000","last_clean":"2023-09-12T14:54:59.806975+0000","last_became_active":"2023-09-12T14:54:58.734886+0000","last_became_peered":"2023-09-12T14:54:58.734886+0000","last_unstale":"2023-09-12T14:54:59.806975+0000","last_undegraded":"2023-09-12T14:54:59.806975+0000","last_fullsized":"2023-09-12T14:54:59.806975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:56:13.426275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.705851+0000","last_change":"2023-09-12T14:54:58.751668+0000","last_active":"2023-09-12T14:54:59.705851+0000","last_peered":"2023-09-12T14:54:59.705851+0000","last_clean":"2023-09-12T14:54:59.705851+0000","last_became_active":"2023-09-12T14:54:58.751016+0000","last_became_peered":"2023-09-12T14:54:58.751016+0000","last_unstale":"2023-09-12T14:54:59.705851+0000","last_undegraded":"2023-09-12T14:54:59.705851+0000","last_fullsized":"2023-09-12T14:54:59.705851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:55:04.268808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806474+0000","last_change":"2023-09-12T14:54:58.749393+0000","last_active":"2023-09-12T14:54:59.806474+0000","last_peered":"2023-09-12T14:54:59.806474+0000","last_clean":"2023-09-12T14:54:59.806474+0000","last_became_active":"2023-09-12T14:54:58.748795+0000","last_became_peered":"2023-09-12T14:54:58.748795+0000","last_unstale":"2023-09-12T14:54:59.806474+0000","last_undegraded":"2023-09-12T14:54:59.806474+0000","last_fullsized":"2023-09-12T14:54:59.806474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:18:31.201019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.705919+0000","last_change":"2023-09-12T14:54:58.722071+0000","last_active":"2023-09-12T14:54:59.705919+0000","last_peered":"2023-09-12T14:54:59.705919+0000","last_clean":"2023-09-12T14:54:59.705919+0000","last_became_active":"2023-09-12T14:54:58.721356+0000","last_became_peered":"2023-09-12T14:54:58.721356+0000","last_unstale":"2023-09-12T14:54:59.705919+0000","last_undegraded":"2023-09-12T14:54:59.705919+0000","last_fullsized":"2023-09-12T14:54:59.705919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:26:45.799536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806581+0000","last_change":"2023-09-12T14:54:58.762364+0000","last_active":"2023-09-12T14:54:59.806581+0000","last_peered":"2023-09-12T14:54:59.806581+0000","last_clean":"2023-09-12T14:54:59.806581+0000","last_became_active":"2023-09-12T14:54:58.761809+0000","last_became_peered":"2023-09-12T14:54:58.761809+0000","last_unstale":"2023-09-12T14:54:59.806581+0000","last_undegraded":"2023-09-12T14:54:59.806581+0000","last_fullsized":"2023-09-12T14:54:59.806581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:55:04.268808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807929+0000","last_change":"2023-09-12T14:54:58.749962+0000","last_active":"2023-09-12T14:54:59.807929+0000","last_peered":"2023-09-12T14:54:59.807929+0000","last_clean":"2023-09-12T14:54:59.807929+0000","last_became_active":"2023-09-12T14:54:58.749252+0000","last_became_peered":"2023-09-12T14:54:58.749252+0000","last_unstale":"2023-09-12T14:54:59.807929+0000","last_undegraded":"2023-09-12T14:54:59.807929+0000","last_fullsized":"2023-09-12T14:54:59.807929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:58:08.195627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807390+0000","last_change":"2023-09-12T14:54:58.745069+0000","last_active":"2023-09-12T14:54:59.807390+0000","last_peered":"2023-09-12T14:54:59.807390+0000","last_clean":"2023-09-12T14:54:59.807390+0000","last_became_active":"2023-09-12T14:54:58.744344+0000","last_became_peered":"2023-09-12T14:54:58.744344+0000","last_unstale":"2023-09-12T14:54:59.807390+0000","last_undegraded":"2023-09-12T14:54:59.807390+0000","last_fullsized":"2023-09-12T14:54:59.807390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:07:30.650142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807913+0000","last_change":"2023-09-12T14:54:58.746751+0000","last_active":"2023-09-12T14:54:59.807913+0000","last_peered":"2023-09-12T14:54:59.807913+0000","last_clean":"2023-09-12T14:54:59.807913+0000","last_became_active":"2023-09-12T14:54:58.746118+0000","last_became_peered":"2023-09-12T14:54:58.746118+0000","last_unstale":"2023-09-12T14:54:59.807913+0000","last_undegraded":"2023-09-12T14:54:59.807913+0000","last_fullsized":"2023-09-12T14:54:59.807913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:18:31.201019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807341+0000","last_change":"2023-09-12T14:54:58.744115+0000","last_active":"2023-09-12T14:54:59.807341+0000","last_peered":"2023-09-12T14:54:59.807341+0000","last_clean":"2023-09-12T14:54:59.807341+0000","last_became_active":"2023-09-12T14:54:58.743477+0000","last_became_peered":"2023-09-12T14:54:58.743477+0000","last_unstale":"2023-09-12T14:54:59.807341+0000","last_undegraded":"2023-09-12T14:54:59.807341+0000","last_fullsized":"2023-09-12T14:54:59.807341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:33.647832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.709077+0000","last_change":"2023-09-12T14:54:58.750301+0000","last_active":"2023-09-12T14:54:59.709077+0000","last_peered":"2023-09-12T14:54:59.709077+0000","last_clean":"2023-09-12T14:54:59.709077+0000","last_became_active":"2023-09-12T14:54:58.748251+0000","last_became_peered":"2023-09-12T14:54:58.748251+0000","last_unstale":"2023-09-12T14:54:59.709077+0000","last_undegraded":"2023-09-12T14:54:59.709077+0000","last_fullsized":"2023-09-12T14:54:59.709077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:35.965864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808454+0000","last_change":"2023-09-12T14:54:58.750934+0000","last_active":"2023-09-12T14:54:59.808454+0000","last_peered":"2023-09-12T14:54:59.808454+0000","last_clean":"2023-09-12T14:54:59.808454+0000","last_became_active":"2023-09-12T14:54:58.750202+0000","last_became_peered":"2023-09-12T14:54:58.750202+0000","last_unstale":"2023-09-12T14:54:59.808454+0000","last_undegraded":"2023-09-12T14:54:59.808454+0000","last_fullsized":"2023-09-12T14:54:59.808454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:33.647832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806666+0000","last_change":"2023-09-12T14:54:58.730838+0000","last_active":"2023-09-12T14:54:59.806666+0000","last_peered":"2023-09-12T14:54:59.806666+0000","last_clean":"2023-09-12T14:54:59.806666+0000","last_became_active":"2023-09-12T14:54:58.727571+0000","last_became_peered":"2023-09-12T14:54:58.727571+0000","last_unstale":"2023-09-12T14:54:59.806666+0000","last_undegraded":"2023-09-12T14:54:59.806666+0000","last_fullsized":"2023-09-12T14:54:59.806666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:58:08.195627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.700662+0000","last_change":"2023-09-12T14:54:58.723535+0000","last_active":"2023-09-12T14:54:59.700662+0000","last_peered":"2023-09-12T14:54:59.700662+0000","last_clean":"2023-09-12T14:54:59.700662+0000","last_became_active":"2023-09-12T14:54:58.722775+0000","last_became_peered":"2023-09-12T14:54:58.722775+0000","last_unstale":"2023-09-12T14:54:59.700662+0000","last_undegraded":"2023-09-12T14:54:59.700662+0000","last_fullsized":"2023-09-12T14:54:59.700662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:54:14.557968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.705388+0000","last_change":"2023-09-12T14:54:58.720670+0000","last_active":"2023-09-12T14:54:59.705388+0000","last_peered":"2023-09-12T14:54:59.705388+0000","last_clean":"2023-09-12T14:54:59.705388+0000","last_became_active":"2023-09-12T14:54:58.719787+0000","last_became_peered":"2023-09-12T14:54:58.719787+0000","last_unstale":"2023-09-12T14:54:59.705388+0000","last_undegraded":"2023-09-12T14:54:59.705388+0000","last_fullsized":"2023-09-12T14:54:59.705388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:33:30.961102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.708175+0000","last_change":"2023-09-12T14:54:58.733183+0000","last_active":"2023-09-12T14:54:59.708175+0000","last_peered":"2023-09-12T14:54:59.708175+0000","last_clean":"2023-09-12T14:54:59.708175+0000","last_became_active":"2023-09-12T14:54:58.732383+0000","last_became_peered":"2023-09-12T14:54:58.732383+0000","last_unstale":"2023-09-12T14:54:59.708175+0000","last_undegraded":"2023-09-12T14:54:59.708175+0000","last_fullsized":"2023-09-12T14:54:59.708175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:53:00.606916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805991+0000","last_change":"2023-09-12T14:54:58.751801+0000","last_active":"2023-09-12T14:54:59.805991+0000","last_peered":"2023-09-12T14:54:59.805991+0000","last_clean":"2023-09-12T14:54:59.805991+0000","last_became_active":"2023-09-12T14:54:58.748758+0000","last_became_peered":"2023-09-12T14:54:58.748758+0000","last_unstale":"2023-09-12T14:54:59.805991+0000","last_undegraded":"2023-09-12T14:54:59.805991+0000","last_fullsized":"2023-09-12T14:54:59.805991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:34:39.391000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807256+0000","last_change":"2023-09-12T14:54:58.745955+0000","last_active":"2023-09-12T14:54:59.807256+0000","last_peered":"2023-09-12T14:54:59.807256+0000","last_clean":"2023-09-12T14:54:59.807256+0000","last_became_active":"2023-09-12T14:54:58.744876+0000","last_became_peered":"2023-09-12T14:54:58.744876+0000","last_unstale":"2023-09-12T14:54:59.807256+0000","last_undegraded":"2023-09-12T14:54:59.807256+0000","last_fullsized":"2023-09-12T14:54:59.807256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:13:39.293184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807685+0000","last_change":"2023-09-12T14:54:58.744120+0000","last_active":"2023-09-12T14:54:59.807685+0000","last_peered":"2023-09-12T14:54:59.807685+0000","last_clean":"2023-09-12T14:54:59.807685+0000","last_became_active":"2023-09-12T14:54:58.743538+0000","last_became_peered":"2023-09-12T14:54:58.743538+0000","last_unstale":"2023-09-12T14:54:59.807685+0000","last_undegraded":"2023-09-12T14:54:59.807685+0000","last_fullsized":"2023-09-12T14:54:59.807685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:10:09.339305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806423+0000","last_change":"2023-09-12T14:54:58.733606+0000","last_active":"2023-09-12T14:54:59.806423+0000","last_peered":"2023-09-12T14:54:59.806423+0000","last_clean":"2023-09-12T14:54:59.806423+0000","last_became_active":"2023-09-12T14:54:58.732811+0000","last_became_peered":"2023-09-12T14:54:58.732811+0000","last_unstale":"2023-09-12T14:54:59.806423+0000","last_undegraded":"2023-09-12T14:54:59.806423+0000","last_fullsized":"2023-09-12T14:54:59.806423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:25:02.224632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706033+0000","last_change":"2023-09-12T14:54:58.733719+0000","last_active":"2023-09-12T14:54:59.706033+0000","last_peered":"2023-09-12T14:54:59.706033+0000","last_clean":"2023-09-12T14:54:59.706033+0000","last_became_active":"2023-09-12T14:54:58.733053+0000","last_became_peered":"2023-09-12T14:54:58.733053+0000","last_unstale":"2023-09-12T14:54:59.706033+0000","last_undegraded":"2023-09-12T14:54:59.706033+0000","last_fullsized":"2023-09-12T14:54:59.706033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:16:18.487363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808066+0000","last_change":"2023-09-12T14:54:58.744857+0000","last_active":"2023-09-12T14:54:59.808066+0000","last_peered":"2023-09-12T14:54:59.808066+0000","last_clean":"2023-09-12T14:54:59.808066+0000","last_became_active":"2023-09-12T14:54:58.744250+0000","last_became_peered":"2023-09-12T14:54:58.744250+0000","last_unstale":"2023-09-12T14:54:59.808066+0000","last_undegraded":"2023-09-12T14:54:59.808066+0000","last_fullsized":"2023-09-12T14:54:59.808066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:09:24.488408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.705573+0000","last_change":"2023-09-12T14:54:58.751371+0000","last_active":"2023-09-12T14:54:59.705573+0000","last_peered":"2023-09-12T14:54:59.705573+0000","last_clean":"2023-09-12T14:54:59.705573+0000","last_became_active":"2023-09-12T14:54:58.750585+0000","last_became_peered":"2023-09-12T14:54:58.750585+0000","last_unstale":"2023-09-12T14:54:59.705573+0000","last_undegraded":"2023-09-12T14:54:59.705573+0000","last_fullsized":"2023-09-12T14:54:59.705573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:18:31.201019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.809302+0000","last_change":"2023-09-12T14:54:58.745970+0000","last_active":"2023-09-12T14:54:59.809302+0000","last_peered":"2023-09-12T14:54:59.809302+0000","last_clean":"2023-09-12T14:54:59.809302+0000","last_became_active":"2023-09-12T14:54:58.745163+0000","last_became_peered":"2023-09-12T14:54:58.745163+0000","last_unstale":"2023-09-12T14:54:59.809302+0000","last_undegraded":"2023-09-12T14:54:59.809302+0000","last_fullsized":"2023-09-12T14:54:59.809302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:55:04.268808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807739+0000","last_change":"2023-09-12T14:54:58.746782+0000","last_active":"2023-09-12T14:54:59.807739+0000","last_peered":"2023-09-12T14:54:59.807739+0000","last_clean":"2023-09-12T14:54:59.807739+0000","last_became_active":"2023-09-12T14:54:58.746149+0000","last_became_peered":"2023-09-12T14:54:58.746149+0000","last_unstale":"2023-09-12T14:54:59.807739+0000","last_undegraded":"2023-09-12T14:54:59.807739+0000","last_fullsized":"2023-09-12T14:54:59.807739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:33:30.961102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.701439+0000","last_change":"2023-09-12T14:54:58.747221+0000","last_active":"2023-09-12T14:54:59.701439+0000","last_peered":"2023-09-12T14:54:59.701439+0000","last_clean":"2023-09-12T14:54:59.701439+0000","last_became_active":"2023-09-12T14:54:58.746509+0000","last_became_peered":"2023-09-12T14:54:58.746509+0000","last_unstale":"2023-09-12T14:54:59.701439+0000","last_undegraded":"2023-09-12T14:54:59.701439+0000","last_fullsized":"2023-09-12T14:54:59.701439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:33.647832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808902+0000","last_change":"2023-09-12T14:54:58.754523+0000","last_active":"2023-09-12T14:54:59.808902+0000","last_peered":"2023-09-12T14:54:59.808902+0000","last_clean":"2023-09-12T14:54:59.808902+0000","last_became_active":"2023-09-12T14:54:58.753849+0000","last_became_peered":"2023-09-12T14:54:58.753849+0000","last_unstale":"2023-09-12T14:54:59.808902+0000","last_undegraded":"2023-09-12T14:54:59.808902+0000","last_fullsized":"2023-09-12T14:54:59.808902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:18:31.201019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806568+0000","last_change":"2023-09-12T14:54:58.750572+0000","last_active":"2023-09-12T14:54:59.806568+0000","last_peered":"2023-09-12T14:54:59.806568+0000","last_clean":"2023-09-12T14:54:59.806568+0000","last_became_active":"2023-09-12T14:54:58.750004+0000","last_became_peered":"2023-09-12T14:54:58.750004+0000","last_unstale":"2023-09-12T14:54:59.806568+0000","last_undegraded":"2023-09-12T14:54:59.806568+0000","last_fullsized":"2023-09-12T14:54:59.806568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:54:14.557968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706258+0000","last_change":"2023-09-12T14:54:58.754746+0000","last_active":"2023-09-12T14:54:59.706258+0000","last_peered":"2023-09-12T14:54:59.706258+0000","last_clean":"2023-09-12T14:54:59.706258+0000","last_became_active":"2023-09-12T14:54:58.754081+0000","last_became_peered":"2023-09-12T14:54:58.754081+0000","last_unstale":"2023-09-12T14:54:59.706258+0000","last_undegraded":"2023-09-12T14:54:59.706258+0000","last_fullsized":"2023-09-12T14:54:59.706258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:23:41.038660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805117+0000","last_change":"2023-09-12T14:54:58.739330+0000","last_active":"2023-09-12T14:54:59.805117+0000","last_peered":"2023-09-12T14:54:59.805117+0000","last_clean":"2023-09-12T14:54:59.805117+0000","last_became_active":"2023-09-12T14:54:58.738451+0000","last_became_peered":"2023-09-12T14:54:58.738451+0000","last_unstale":"2023-09-12T14:54:59.805117+0000","last_undegraded":"2023-09-12T14:54:59.805117+0000","last_fullsized":"2023-09-12T14:54:59.805117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:35.965864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805898+0000","last_change":"2023-09-12T14:54:58.745424+0000","last_active":"2023-09-12T14:54:59.805898+0000","last_peered":"2023-09-12T14:54:59.805898+0000","last_clean":"2023-09-12T14:54:59.805898+0000","last_became_active":"2023-09-12T14:54:58.744842+0000","last_became_peered":"2023-09-12T14:54:58.744842+0000","last_unstale":"2023-09-12T14:54:59.805898+0000","last_undegraded":"2023-09-12T14:54:59.805898+0000","last_fullsized":"2023-09-12T14:54:59.805898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:09:24.488408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.708763+0000","last_change":"2023-09-12T14:54:58.734734+0000","last_active":"2023-09-12T14:54:59.708763+0000","last_peered":"2023-09-12T14:54:59.708763+0000","last_clean":"2023-09-12T14:54:59.708763+0000","last_became_active":"2023-09-12T14:54:58.730411+0000","last_became_peered":"2023-09-12T14:54:58.730411+0000","last_unstale":"2023-09-12T14:54:59.708763+0000","last_undegraded":"2023-09-12T14:54:59.708763+0000","last_fullsized":"2023-09-12T14:54:59.708763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:04:39.183616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808747+0000","last_change":"2023-09-12T14:54:57.709617+0000","last_active":"2023-09-12T14:54:59.808747+0000","last_peered":"2023-09-12T14:54:59.808747+0000","last_clean":"2023-09-12T14:54:59.808747+0000","last_became_active":"2023-09-12T14:54:57.709171+0000","last_became_peered":"2023-09-12T14:54:57.709171+0000","last_unstale":"2023-09-12T14:54:59.808747+0000","last_undegraded":"2023-09-12T14:54:59.808747+0000","last_fullsized":"2023-09-12T14:54:59.808747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:35:05.562961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806849+0000","last_change":"2023-09-12T14:54:58.722571+0000","last_active":"2023-09-12T14:54:59.806849+0000","last_peered":"2023-09-12T14:54:59.806849+0000","last_clean":"2023-09-12T14:54:59.806849+0000","last_became_active":"2023-09-12T14:54:58.721782+0000","last_became_peered":"2023-09-12T14:54:58.721782+0000","last_unstale":"2023-09-12T14:54:59.806849+0000","last_undegraded":"2023-09-12T14:54:59.806849+0000","last_fullsized":"2023-09-12T14:54:59.806849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:22:58.799616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807614+0000","last_change":"2023-09-12T14:54:57.709417+0000","last_active":"2023-09-12T14:54:59.807614+0000","last_peered":"2023-09-12T14:54:59.807614+0000","last_clean":"2023-09-12T14:54:59.807614+0000","last_became_active":"2023-09-12T14:54:57.708756+0000","last_became_peered":"2023-09-12T14:54:57.708756+0000","last_unstale":"2023-09-12T14:54:59.807614+0000","last_undegraded":"2023-09-12T14:54:59.807614+0000","last_fullsized":"2023-09-12T14:54:59.807614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:50:58.241369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808240+0000","last_change":"2023-09-12T14:54:57.698971+0000","last_active":"2023-09-12T14:54:59.808240+0000","last_peered":"2023-09-12T14:54:59.808240+0000","last_clean":"2023-09-12T14:54:59.808240+0000","last_became_active":"2023-09-12T14:54:57.698574+0000","last_became_peered":"2023-09-12T14:54:57.698574+0000","last_unstale":"2023-09-12T14:54:59.808240+0000","last_undegraded":"2023-09-12T14:54:59.808240+0000","last_fullsized":"2023-09-12T14:54:59.808240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:25:01.213694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808091+0000","last_change":"2023-09-12T14:54:58.748244+0000","last_active":"2023-09-12T14:54:59.808091+0000","last_peered":"2023-09-12T14:54:59.808091+0000","last_clean":"2023-09-12T14:54:59.808091+0000","last_became_active":"2023-09-12T14:54:58.746823+0000","last_became_peered":"2023-09-12T14:54:58.746823+0000","last_unstale":"2023-09-12T14:54:59.808091+0000","last_undegraded":"2023-09-12T14:54:59.808091+0000","last_fullsized":"2023-09-12T14:54:59.808091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:22:58.799616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706293+0000","last_change":"2023-09-12T14:54:57.708138+0000","last_active":"2023-09-12T14:54:59.706293+0000","last_peered":"2023-09-12T14:54:59.706293+0000","last_clean":"2023-09-12T14:54:59.706293+0000","last_became_active":"2023-09-12T14:54:57.705380+0000","last_became_peered":"2023-09-12T14:54:57.705380+0000","last_unstale":"2023-09-12T14:54:59.706293+0000","last_undegraded":"2023-09-12T14:54:59.706293+0000","last_fullsized":"2023-09-12T14:54:59.706293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:34:38.380062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.700719+0000","last_change":"2023-09-12T14:54:58.751072+0000","last_active":"2023-09-12T14:54:59.700719+0000","last_peered":"2023-09-12T14:54:59.700719+0000","last_clean":"2023-09-12T14:54:59.700719+0000","last_became_active":"2023-09-12T14:54:58.750309+0000","last_became_peered":"2023-09-12T14:54:58.750309+0000","last_unstale":"2023-09-12T14:54:59.700719+0000","last_undegraded":"2023-09-12T14:54:59.700719+0000","last_fullsized":"2023-09-12T14:54:59.700719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:58:08.195627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808058+0000","last_change":"2023-09-12T14:54:57.719821+0000","last_active":"2023-09-12T14:54:59.808058+0000","last_peered":"2023-09-12T14:54:59.808058+0000","last_clean":"2023-09-12T14:54:59.808058+0000","last_became_active":"2023-09-12T14:54:57.717672+0000","last_became_peered":"2023-09-12T14:54:57.717672+0000","last_unstale":"2023-09-12T14:54:59.808058+0000","last_undegraded":"2023-09-12T14:54:59.808058+0000","last_fullsized":"2023-09-12T14:54:59.808058+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:25:01.213694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.707290+0000","last_change":"2023-09-12T14:54:58.745459+0000","last_active":"2023-09-12T14:54:59.707290+0000","last_peered":"2023-09-12T14:54:59.707290+0000","last_clean":"2023-09-12T14:54:59.707290+0000","last_became_active":"2023-09-12T14:54:58.744478+0000","last_became_peered":"2023-09-12T14:54:58.744478+0000","last_unstale":"2023-09-12T14:54:59.707290+0000","last_undegraded":"2023-09-12T14:54:59.707290+0000","last_fullsized":"2023-09-12T14:54:59.707290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:07:30.650142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.707357+0000","last_change":"2023-09-12T14:54:57.712325+0000","last_active":"2023-09-12T14:54:59.707357+0000","last_peered":"2023-09-12T14:54:59.707357+0000","last_clean":"2023-09-12T14:54:59.707357+0000","last_became_active":"2023-09-12T14:54:57.711669+0000","last_became_peered":"2023-09-12T14:54:57.711669+0000","last_unstale":"2023-09-12T14:54:59.707357+0000","last_undegraded":"2023-09-12T14:54:59.707357+0000","last_fullsized":"2023-09-12T14:54:59.707357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:25:01.213694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807090+0000","last_change":"2023-09-12T14:54:58.731110+0000","last_active":"2023-09-12T14:54:59.807090+0000","last_peered":"2023-09-12T14:54:59.807090+0000","last_clean":"2023-09-12T14:54:59.807090+0000","last_became_active":"2023-09-12T14:54:58.728351+0000","last_became_peered":"2023-09-12T14:54:58.728351+0000","last_unstale":"2023-09-12T14:54:59.807090+0000","last_undegraded":"2023-09-12T14:54:59.807090+0000","last_fullsized":"2023-09-12T14:54:59.807090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:10:09.339305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.700987+0000","last_change":"2023-09-12T14:54:57.699372+0000","last_active":"2023-09-12T14:54:59.700987+0000","last_peered":"2023-09-12T14:54:59.700987+0000","last_clean":"2023-09-12T14:54:59.700987+0000","last_became_active":"2023-09-12T14:54:57.698970+0000","last_became_peered":"2023-09-12T14:54:57.698970+0000","last_unstale":"2023-09-12T14:54:59.700987+0000","last_undegraded":"2023-09-12T14:54:59.700987+0000","last_fullsized":"2023-09-12T14:54:59.700987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:50:58.241369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.709034+0000","last_change":"2023-09-12T14:54:58.753601+0000","last_active":"2023-09-12T14:54:59.709034+0000","last_peered":"2023-09-12T14:54:59.709034+0000","last_clean":"2023-09-12T14:54:59.709034+0000","last_became_active":"2023-09-12T14:54:58.752414+0000","last_became_peered":"2023-09-12T14:54:58.752414+0000","last_unstale":"2023-09-12T14:54:59.709034+0000","last_undegraded":"2023-09-12T14:54:59.709034+0000","last_fullsized":"2023-09-12T14:54:59.709034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:41:25.126595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807970+0000","last_change":"2023-09-12T14:54:57.699723+0000","last_active":"2023-09-12T14:54:59.807970+0000","last_peered":"2023-09-12T14:54:59.807970+0000","last_clean":"2023-09-12T14:54:59.807970+0000","last_became_active":"2023-09-12T14:54:57.698942+0000","last_became_peered":"2023-09-12T14:54:57.698942+0000","last_unstale":"2023-09-12T14:54:59.807970+0000","last_undegraded":"2023-09-12T14:54:59.807970+0000","last_fullsized":"2023-09-12T14:54:59.807970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:35:05.562961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806782+0000","last_change":"2023-09-12T14:54:58.756683+0000","last_active":"2023-09-12T14:54:59.806782+0000","last_peered":"2023-09-12T14:54:59.806782+0000","last_clean":"2023-09-12T14:54:59.806782+0000","last_became_active":"2023-09-12T14:54:58.755952+0000","last_became_peered":"2023-09-12T14:54:58.755952+0000","last_unstale":"2023-09-12T14:54:59.806782+0000","last_undegraded":"2023-09-12T14:54:59.806782+0000","last_fullsized":"2023-09-12T14:54:59.806782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:07:26.691746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806819+0000","last_change":"2023-09-12T14:54:57.708009+0000","last_active":"2023-09-12T14:54:59.806819+0000","last_peered":"2023-09-12T14:54:59.806819+0000","last_clean":"2023-09-12T14:54:59.806819+0000","last_became_active":"2023-09-12T14:54:57.707617+0000","last_became_peered":"2023-09-12T14:54:57.707617+0000","last_unstale":"2023-09-12T14:54:59.806819+0000","last_undegraded":"2023-09-12T14:54:59.806819+0000","last_fullsized":"2023-09-12T14:54:59.806819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:10:06.188199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806973+0000","last_change":"2023-09-12T14:54:58.747555+0000","last_active":"2023-09-12T14:54:59.806973+0000","last_peered":"2023-09-12T14:54:59.806973+0000","last_clean":"2023-09-12T14:54:59.806973+0000","last_became_active":"2023-09-12T14:54:58.745580+0000","last_became_peered":"2023-09-12T14:54:58.745580+0000","last_unstale":"2023-09-12T14:54:59.806973+0000","last_undegraded":"2023-09-12T14:54:59.806973+0000","last_fullsized":"2023-09-12T14:54:59.806973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:33.647832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706246+0000","last_change":"2023-09-12T14:54:57.713557+0000","last_active":"2023-09-12T14:54:59.706246+0000","last_peered":"2023-09-12T14:54:59.706246+0000","last_clean":"2023-09-12T14:54:59.706246+0000","last_became_active":"2023-09-12T14:54:57.712038+0000","last_became_peered":"2023-09-12T14:54:57.712038+0000","last_unstale":"2023-09-12T14:54:59.706246+0000","last_undegraded":"2023-09-12T14:54:59.706246+0000","last_fullsized":"2023-09-12T14:54:59.706246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:22:57.788678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806857+0000","last_change":"2023-09-12T14:54:58.754016+0000","last_active":"2023-09-12T14:54:59.806857+0000","last_peered":"2023-09-12T14:54:59.806857+0000","last_clean":"2023-09-12T14:54:59.806857+0000","last_became_active":"2023-09-12T14:54:58.753380+0000","last_became_peered":"2023-09-12T14:54:58.753380+0000","last_unstale":"2023-09-12T14:54:59.806857+0000","last_undegraded":"2023-09-12T14:54:59.806857+0000","last_fullsized":"2023-09-12T14:54:59.806857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:16:18.487363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805823+0000","last_change":"2023-09-12T14:54:57.713543+0000","last_active":"2023-09-12T14:54:59.805823+0000","last_peered":"2023-09-12T14:54:59.805823+0000","last_clean":"2023-09-12T14:54:59.805823+0000","last_became_active":"2023-09-12T14:54:57.712819+0000","last_became_peered":"2023-09-12T14:54:57.712819+0000","last_unstale":"2023-09-12T14:54:59.805823+0000","last_undegraded":"2023-09-12T14:54:59.805823+0000","last_fullsized":"2023-09-12T14:54:59.805823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:10:06.188199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805969+0000","last_change":"2023-09-12T14:54:58.745224+0000","last_active":"2023-09-12T14:54:59.805969+0000","last_peered":"2023-09-12T14:54:59.805969+0000","last_clean":"2023-09-12T14:54:59.805969+0000","last_became_active":"2023-09-12T14:54:58.744566+0000","last_became_peered":"2023-09-12T14:54:58.744566+0000","last_unstale":"2023-09-12T14:54:59.805969+0000","last_undegraded":"2023-09-12T14:54:59.805969+0000","last_fullsized":"2023-09-12T14:54:59.805969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:56:13.426275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805117+0000","last_change":"2023-09-12T14:54:57.683795+0000","last_active":"2023-09-12T14:54:59.805117+0000","last_peered":"2023-09-12T14:54:59.805117+0000","last_clean":"2023-09-12T14:54:59.805117+0000","last_became_active":"2023-09-12T14:54:57.683309+0000","last_became_peered":"2023-09-12T14:54:57.683309+0000","last_unstale":"2023-09-12T14:54:59.805117+0000","last_undegraded":"2023-09-12T14:54:59.805117+0000","last_fullsized":"2023-09-12T14:54:59.805117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:38:46.256403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806420+0000","last_change":"2023-09-12T14:54:58.744197+0000","last_active":"2023-09-12T14:54:59.806420+0000","last_peered":"2023-09-12T14:54:59.806420+0000","last_clean":"2023-09-12T14:54:59.806420+0000","last_became_active":"2023-09-12T14:54:58.743668+0000","last_became_peered":"2023-09-12T14:54:58.743668+0000","last_unstale":"2023-09-12T14:54:59.806420+0000","last_undegraded":"2023-09-12T14:54:59.806420+0000","last_fullsized":"2023-09-12T14:54:59.806420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:33:30.961102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.709354+0000","last_change":"2023-09-12T14:54:57.704349+0000","last_active":"2023-09-12T14:54:59.709354+0000","last_peered":"2023-09-12T14:54:59.709354+0000","last_clean":"2023-09-12T14:54:59.709354+0000","last_became_active":"2023-09-12T14:54:57.703134+0000","last_became_peered":"2023-09-12T14:54:57.703134+0000","last_unstale":"2023-09-12T14:54:59.709354+0000","last_undegraded":"2023-09-12T14:54:59.709354+0000","last_fullsized":"2023-09-12T14:54:59.709354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:25:54.071276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807199+0000","last_change":"2023-09-12T14:54:58.752753+0000","last_active":"2023-09-12T14:54:59.807199+0000","last_peered":"2023-09-12T14:54:59.807199+0000","last_clean":"2023-09-12T14:54:59.807199+0000","last_became_active":"2023-09-12T14:54:58.751852+0000","last_became_peered":"2023-09-12T14:54:58.751852+0000","last_unstale":"2023-09-12T14:54:59.807199+0000","last_undegraded":"2023-09-12T14:54:59.807199+0000","last_fullsized":"2023-09-12T14:54:59.807199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:23:41.038660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806233+0000","last_change":"2023-09-12T14:54:57.691156+0000","last_active":"2023-09-12T14:54:59.806233+0000","last_peered":"2023-09-12T14:54:59.806233+0000","last_clean":"2023-09-12T14:54:59.806233+0000","last_became_active":"2023-09-12T14:54:57.690769+0000","last_became_peered":"2023-09-12T14:54:57.690769+0000","last_unstale":"2023-09-12T14:54:59.806233+0000","last_undegraded":"2023-09-12T14:54:59.806233+0000","last_fullsized":"2023-09-12T14:54:59.806233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:48:28.111452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807364+0000","last_change":"2023-09-12T14:54:58.755613+0000","last_active":"2023-09-12T14:54:59.807364+0000","last_peered":"2023-09-12T14:54:59.807364+0000","last_clean":"2023-09-12T14:54:59.807364+0000","last_became_active":"2023-09-12T14:54:58.755005+0000","last_became_peered":"2023-09-12T14:54:58.755005+0000","last_unstale":"2023-09-12T14:54:59.807364+0000","last_undegraded":"2023-09-12T14:54:59.807364+0000","last_fullsized":"2023-09-12T14:54:59.807364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:27:30.157653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.701127+0000","last_change":"2023-09-12T14:54:57.686468+0000","last_active":"2023-09-12T14:54:59.701127+0000","last_peered":"2023-09-12T14:54:59.701127+0000","last_clean":"2023-09-12T14:54:59.701127+0000","last_became_active":"2023-09-12T14:54:57.686059+0000","last_became_peered":"2023-09-12T14:54:57.686059+0000","last_unstale":"2023-09-12T14:54:59.701127+0000","last_undegraded":"2023-09-12T14:54:59.701127+0000","last_fullsized":"2023-09-12T14:54:59.701127+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:35:05.562961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706190+0000","last_change":"2023-09-12T14:54:58.751002+0000","last_active":"2023-09-12T14:54:59.706190+0000","last_peered":"2023-09-12T14:54:59.706190+0000","last_clean":"2023-09-12T14:54:59.706190+0000","last_became_active":"2023-09-12T14:54:58.750380+0000","last_became_peered":"2023-09-12T14:54:58.750380+0000","last_unstale":"2023-09-12T14:54:59.706190+0000","last_undegraded":"2023-09-12T14:54:59.706190+0000","last_fullsized":"2023-09-12T14:54:59.706190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:56:32.731297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805597+0000","last_change":"2023-09-12T14:54:57.680566+0000","last_active":"2023-09-12T14:54:59.805597+0000","last_peered":"2023-09-12T14:54:59.805597+0000","last_clean":"2023-09-12T14:54:59.805597+0000","last_became_active":"2023-09-12T14:54:57.680113+0000","last_became_peered":"2023-09-12T14:54:57.680113+0000","last_unstale":"2023-09-12T14:54:59.805597+0000","last_undegraded":"2023-09-12T14:54:59.805597+0000","last_fullsized":"2023-09-12T14:54:59.805597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:48:51.271671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806912+0000","last_change":"2023-09-12T14:54:58.746352+0000","last_active":"2023-09-12T14:54:59.806912+0000","last_peered":"2023-09-12T14:54:59.806912+0000","last_clean":"2023-09-12T14:54:59.806912+0000","last_became_active":"2023-09-12T14:54:58.745469+0000","last_became_peered":"2023-09-12T14:54:58.745469+0000","last_unstale":"2023-09-12T14:54:59.806912+0000","last_undegraded":"2023-09-12T14:54:59.806912+0000","last_fullsized":"2023-09-12T14:54:59.806912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:56:32.731297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-12T14:54:58.722614+0000","last_change":"2023-09-12T14:54:46.965446+0000","last_active":"2023-09-12T14:54:58.722614+0000","last_peered":"2023-09-12T14:54:58.722614+0000","last_clean":"2023-09-12T14:54:58.722614+0000","last_became_active":"2023-09-12T14:54:46.963236+0000","last_became_peered":"2023-09-12T14:54:46.963236+0000","last_unstale":"2023-09-12T14:54:58.722614+0000","last_undegraded":"2023-09-12T14:54:58.722614+0000","last_fullsized":"2023-09-12T14:54:58.722614+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:45.804232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:45.804232+0000","last_clean_scrub_stamp":"2023-09-12T14:54:45.804232+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:24:45.670513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.705459+0000","last_change":"2023-09-12T14:54:57.707645+0000","last_active":"2023-09-12T14:54:59.705459+0000","last_peered":"2023-09-12T14:54:59.705459+0000","last_clean":"2023-09-12T14:54:59.705459+0000","last_became_active":"2023-09-12T14:54:57.705665+0000","last_became_peered":"2023-09-12T14:54:57.705665+0000","last_unstale":"2023-09-12T14:54:59.705459+0000","last_undegraded":"2023-09-12T14:54:59.705459+0000","last_fullsized":"2023-09-12T14:54:59.705459+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:48:28.111452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808625+0000","last_change":"2023-09-12T14:54:58.745179+0000","last_active":"2023-09-12T14:54:59.808625+0000","last_peered":"2023-09-12T14:54:59.808625+0000","last_clean":"2023-09-12T14:54:59.808625+0000","last_became_active":"2023-09-12T14:54:58.744548+0000","last_became_peered":"2023-09-12T14:54:58.744548+0000","last_unstale":"2023-09-12T14:54:59.808625+0000","last_undegraded":"2023-09-12T14:54:59.808625+0000","last_fullsized":"2023-09-12T14:54:59.808625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:10:09.339305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806036+0000","last_change":"2023-09-12T14:54:57.719515+0000","last_active":"2023-09-12T14:54:59.806036+0000","last_peered":"2023-09-12T14:54:59.806036+0000","last_clean":"2023-09-12T14:54:59.806036+0000","last_became_active":"2023-09-12T14:54:57.717582+0000","last_became_peered":"2023-09-12T14:54:57.717582+0000","last_unstale":"2023-09-12T14:54:59.806036+0000","last_undegraded":"2023-09-12T14:54:59.806036+0000","last_fullsized":"2023-09-12T14:54:59.806036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:48:51.271671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806471+0000","last_change":"2023-09-12T14:54:58.715137+0000","last_active":"2023-09-12T14:54:59.806471+0000","last_peered":"2023-09-12T14:54:59.806471+0000","last_clean":"2023-09-12T14:54:59.806471+0000","last_became_active":"2023-09-12T14:54:58.713883+0000","last_became_peered":"2023-09-12T14:54:58.713883+0000","last_unstale":"2023-09-12T14:54:59.806471+0000","last_undegraded":"2023-09-12T14:54:59.806471+0000","last_fullsized":"2023-09-12T14:54:59.806471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:20:59.118588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805239+0000","last_change":"2023-09-12T14:54:57.691863+0000","last_active":"2023-09-12T14:54:59.805239+0000","last_peered":"2023-09-12T14:54:59.805239+0000","last_clean":"2023-09-12T14:54:59.805239+0000","last_became_active":"2023-09-12T14:54:57.691453+0000","last_became_peered":"2023-09-12T14:54:57.691453+0000","last_unstale":"2023-09-12T14:54:59.805239+0000","last_undegraded":"2023-09-12T14:54:59.805239+0000","last_fullsized":"2023-09-12T14:54:59.805239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:01:34.927854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.700716+0000","last_change":"2023-09-12T14:54:58.732432+0000","last_active":"2023-09-12T14:54:59.700716+0000","last_peered":"2023-09-12T14:54:59.700716+0000","last_clean":"2023-09-12T14:54:59.700716+0000","last_became_active":"2023-09-12T14:54:58.731662+0000","last_became_peered":"2023-09-12T14:54:58.731662+0000","last_unstale":"2023-09-12T14:54:59.700716+0000","last_undegraded":"2023-09-12T14:54:59.700716+0000","last_fullsized":"2023-09-12T14:54:59.700716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:09:24.488408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706495+0000","last_change":"2023-09-12T14:54:57.714059+0000","last_active":"2023-09-12T14:54:59.706495+0000","last_peered":"2023-09-12T14:54:59.706495+0000","last_clean":"2023-09-12T14:54:59.706495+0000","last_became_active":"2023-09-12T14:54:57.712854+0000","last_became_peered":"2023-09-12T14:54:57.712854+0000","last_unstale":"2023-09-12T14:54:59.706495+0000","last_undegraded":"2023-09-12T14:54:59.706495+0000","last_fullsized":"2023-09-12T14:54:59.706495+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:56:12.415337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.709032+0000","last_change":"2023-09-12T14:54:58.713195+0000","last_active":"2023-09-12T14:54:59.709032+0000","last_peered":"2023-09-12T14:54:59.709032+0000","last_clean":"2023-09-12T14:54:59.709032+0000","last_became_active":"2023-09-12T14:54:58.712392+0000","last_became_peered":"2023-09-12T14:54:58.712392+0000","last_unstale":"2023-09-12T14:54:59.709032+0000","last_undegraded":"2023-09-12T14:54:59.709032+0000","last_fullsized":"2023-09-12T14:54:59.709032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:01:35.938792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.701021+0000","last_change":"2023-09-12T14:54:57.691816+0000","last_active":"2023-09-12T14:54:59.701021+0000","last_peered":"2023-09-12T14:54:59.701021+0000","last_clean":"2023-09-12T14:54:59.701021+0000","last_became_active":"2023-09-12T14:54:57.691410+0000","last_became_peered":"2023-09-12T14:54:57.691410+0000","last_unstale":"2023-09-12T14:54:59.701021+0000","last_undegraded":"2023-09-12T14:54:59.701021+0000","last_fullsized":"2023-09-12T14:54:59.701021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:01:34.927854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706911+0000","last_change":"2023-09-12T14:54:58.748697+0000","last_active":"2023-09-12T14:54:59.706911+0000","last_peered":"2023-09-12T14:54:59.706911+0000","last_clean":"2023-09-12T14:54:59.706911+0000","last_became_active":"2023-09-12T14:54:58.748072+0000","last_became_peered":"2023-09-12T14:54:58.748072+0000","last_unstale":"2023-09-12T14:54:59.706911+0000","last_undegraded":"2023-09-12T14:54:59.706911+0000","last_fullsized":"2023-09-12T14:54:59.706911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:27:30.157653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807241+0000","last_change":"2023-09-12T14:54:57.718970+0000","last_active":"2023-09-12T14:54:59.807241+0000","last_peered":"2023-09-12T14:54:59.807241+0000","last_clean":"2023-09-12T14:54:59.807241+0000","last_became_active":"2023-09-12T14:54:57.717410+0000","last_became_peered":"2023-09-12T14:54:57.717410+0000","last_unstale":"2023-09-12T14:54:59.807241+0000","last_undegraded":"2023-09-12T14:54:59.807241+0000","last_fullsized":"2023-09-12T14:54:59.807241+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:22:57.788678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.701025+0000","last_change":"2023-09-12T14:54:58.733797+0000","last_active":"2023-09-12T14:54:59.701025+0000","last_peered":"2023-09-12T14:54:59.701025+0000","last_clean":"2023-09-12T14:54:59.701025+0000","last_became_active":"2023-09-12T14:54:58.732869+0000","last_became_peered":"2023-09-12T14:54:58.732869+0000","last_unstale":"2023-09-12T14:54:59.701025+0000","last_undegraded":"2023-09-12T14:54:59.701025+0000","last_fullsized":"2023-09-12T14:54:59.701025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:22:58.799616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.701305+0000","last_change":"2023-09-12T14:54:57.703103+0000","last_active":"2023-09-12T14:54:59.701305+0000","last_peered":"2023-09-12T14:54:59.701305+0000","last_clean":"2023-09-12T14:54:59.701305+0000","last_became_active":"2023-09-12T14:54:57.702706+0000","last_became_peered":"2023-09-12T14:54:57.702706+0000","last_unstale":"2023-09-12T14:54:59.701305+0000","last_undegraded":"2023-09-12T14:54:59.701305+0000","last_fullsized":"2023-09-12T14:54:59.701305+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:34:38.380062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.809481+0000","last_change":"2023-09-12T14:54:58.745944+0000","last_active":"2023-09-12T14:54:59.809481+0000","last_peered":"2023-09-12T14:54:59.809481+0000","last_clean":"2023-09-12T14:54:59.809481+0000","last_became_active":"2023-09-12T14:54:58.744677+0000","last_became_peered":"2023-09-12T14:54:58.744677+0000","last_unstale":"2023-09-12T14:54:59.809481+0000","last_undegraded":"2023-09-12T14:54:59.809481+0000","last_fullsized":"2023-09-12T14:54:59.809481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:26:45.799536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808147+0000","last_change":"2023-09-12T14:54:57.719402+0000","last_active":"2023-09-12T14:54:59.808147+0000","last_peered":"2023-09-12T14:54:59.808147+0000","last_clean":"2023-09-12T14:54:59.808147+0000","last_became_active":"2023-09-12T14:54:57.718815+0000","last_became_peered":"2023-09-12T14:54:57.718815+0000","last_unstale":"2023-09-12T14:54:59.808147+0000","last_undegraded":"2023-09-12T14:54:59.808147+0000","last_fullsized":"2023-09-12T14:54:59.808147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:20:58.107650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807282+0000","last_change":"2023-09-12T14:54:58.732619+0000","last_active":"2023-09-12T14:54:59.807282+0000","last_peered":"2023-09-12T14:54:59.807282+0000","last_clean":"2023-09-12T14:54:59.807282+0000","last_became_active":"2023-09-12T14:54:58.731845+0000","last_became_peered":"2023-09-12T14:54:58.731845+0000","last_unstale":"2023-09-12T14:54:59.807282+0000","last_undegraded":"2023-09-12T14:54:59.807282+0000","last_fullsized":"2023-09-12T14:54:59.807282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:33:30.961102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805781+0000","last_change":"2023-09-12T14:54:57.717947+0000","last_active":"2023-09-12T14:54:59.805781+0000","last_peered":"2023-09-12T14:54:59.805781+0000","last_clean":"2023-09-12T14:54:59.805781+0000","last_became_active":"2023-09-12T14:54:57.716465+0000","last_became_peered":"2023-09-12T14:54:57.716465+0000","last_unstale":"2023-09-12T14:54:59.805781+0000","last_undegraded":"2023-09-12T14:54:59.805781+0000","last_fullsized":"2023-09-12T14:54:59.805781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:01:34.927854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.708729+0000","last_change":"2023-09-12T14:54:58.734846+0000","last_active":"2023-09-12T14:54:59.708729+0000","last_peered":"2023-09-12T14:54:59.708729+0000","last_clean":"2023-09-12T14:54:59.708729+0000","last_became_active":"2023-09-12T14:54:58.733232+0000","last_became_peered":"2023-09-12T14:54:58.733232+0000","last_unstale":"2023-09-12T14:54:59.708729+0000","last_undegraded":"2023-09-12T14:54:59.708729+0000","last_fullsized":"2023-09-12T14:54:59.708729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:48:52.282609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.700759+0000","last_change":"2023-09-12T14:54:57.693075+0000","last_active":"2023-09-12T14:54:59.700759+0000","last_peered":"2023-09-12T14:54:59.700759+0000","last_clean":"2023-09-12T14:54:59.700759+0000","last_became_active":"2023-09-12T14:54:57.692190+0000","last_became_peered":"2023-09-12T14:54:57.692190+0000","last_unstale":"2023-09-12T14:54:59.700759+0000","last_undegraded":"2023-09-12T14:54:59.700759+0000","last_fullsized":"2023-09-12T14:54:59.700759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:41:24.115657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808265+0000","last_change":"2023-09-12T14:54:58.748674+0000","last_active":"2023-09-12T14:54:59.808265+0000","last_peered":"2023-09-12T14:54:59.808265+0000","last_clean":"2023-09-12T14:54:59.808265+0000","last_became_active":"2023-09-12T14:54:58.747570+0000","last_became_peered":"2023-09-12T14:54:58.747570+0000","last_unstale":"2023-09-12T14:54:59.808265+0000","last_undegraded":"2023-09-12T14:54:59.808265+0000","last_fullsized":"2023-09-12T14:54:59.808265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:56:13.426275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.700971+0000","last_change":"2023-09-12T14:54:57.690103+0000","last_active":"2023-09-12T14:54:59.700971+0000","last_peered":"2023-09-12T14:54:59.700971+0000","last_clean":"2023-09-12T14:54:59.700971+0000","last_became_active":"2023-09-12T14:54:57.688945+0000","last_became_peered":"2023-09-12T14:54:57.688945+0000","last_unstale":"2023-09-12T14:54:59.700971+0000","last_undegraded":"2023-09-12T14:54:59.700971+0000","last_fullsized":"2023-09-12T14:54:59.700971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:25:01.213694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808786+0000","last_change":"2023-09-12T14:54:58.744905+0000","last_active":"2023-09-12T14:54:59.808786+0000","last_peered":"2023-09-12T14:54:59.808786+0000","last_clean":"2023-09-12T14:54:59.808786+0000","last_became_active":"2023-09-12T14:54:58.744132+0000","last_became_peered":"2023-09-12T14:54:58.744132+0000","last_unstale":"2023-09-12T14:54:59.808786+0000","last_undegraded":"2023-09-12T14:54:59.808786+0000","last_fullsized":"2023-09-12T14:54:59.808786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:33:30.961102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807246+0000","last_change":"2023-09-12T14:54:57.718508+0000","last_active":"2023-09-12T14:54:59.807246+0000","last_peered":"2023-09-12T14:54:59.807246+0000","last_clean":"2023-09-12T14:54:59.807246+0000","last_became_active":"2023-09-12T14:54:57.716751+0000","last_became_peered":"2023-09-12T14:54:57.716751+0000","last_unstale":"2023-09-12T14:54:59.807246+0000","last_undegraded":"2023-09-12T14:54:59.807246+0000","last_fullsized":"2023-09-12T14:54:59.807246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:34:38.380062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805940+0000","last_change":"2023-09-12T14:54:58.747254+0000","last_active":"2023-09-12T14:54:59.805940+0000","last_peered":"2023-09-12T14:54:59.805940+0000","last_clean":"2023-09-12T14:54:59.805940+0000","last_became_active":"2023-09-12T14:54:58.746726+0000","last_became_peered":"2023-09-12T14:54:58.746726+0000","last_unstale":"2023-09-12T14:54:59.805940+0000","last_undegraded":"2023-09-12T14:54:59.805940+0000","last_fullsized":"2023-09-12T14:54:59.805940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:22:58.799616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706879+0000","last_change":"2023-09-12T14:54:58.720140+0000","last_active":"2023-09-12T14:54:59.706879+0000","last_peered":"2023-09-12T14:54:59.706879+0000","last_clean":"2023-09-12T14:54:59.706879+0000","last_became_active":"2023-09-12T14:54:58.717751+0000","last_became_peered":"2023-09-12T14:54:58.717751+0000","last_unstale":"2023-09-12T14:54:59.706879+0000","last_undegraded":"2023-09-12T14:54:59.706879+0000","last_fullsized":"2023-09-12T14:54:59.706879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:13:39.293184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806375+0000","last_change":"2023-09-12T14:54:57.717851+0000","last_active":"2023-09-12T14:54:59.806375+0000","last_peered":"2023-09-12T14:54:59.806375+0000","last_clean":"2023-09-12T14:54:59.806375+0000","last_became_active":"2023-09-12T14:54:57.716821+0000","last_became_peered":"2023-09-12T14:54:57.716821+0000","last_unstale":"2023-09-12T14:54:59.806375+0000","last_undegraded":"2023-09-12T14:54:59.806375+0000","last_fullsized":"2023-09-12T14:54:59.806375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:34.954926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.705907+0000","last_change":"2023-09-12T14:54:58.722319+0000","last_active":"2023-09-12T14:54:59.705907+0000","last_peered":"2023-09-12T14:54:59.705907+0000","last_clean":"2023-09-12T14:54:59.705907+0000","last_became_active":"2023-09-12T14:54:58.719871+0000","last_became_peered":"2023-09-12T14:54:58.719871+0000","last_unstale":"2023-09-12T14:54:59.705907+0000","last_undegraded":"2023-09-12T14:54:59.705907+0000","last_fullsized":"2023-09-12T14:54:59.705907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:07:26.691746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807030+0000","last_change":"2023-09-12T14:54:57.718179+0000","last_active":"2023-09-12T14:54:59.807030+0000","last_peered":"2023-09-12T14:54:59.807030+0000","last_clean":"2023-09-12T14:54:59.807030+0000","last_became_active":"2023-09-12T14:54:57.716712+0000","last_became_peered":"2023-09-12T14:54:57.716712+0000","last_unstale":"2023-09-12T14:54:59.807030+0000","last_undegraded":"2023-09-12T14:54:59.807030+0000","last_fullsized":"2023-09-12T14:54:59.807030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:25:01.213694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806750+0000","last_change":"2023-09-12T14:54:58.728727+0000","last_active":"2023-09-12T14:54:59.806750+0000","last_peered":"2023-09-12T14:54:59.806750+0000","last_clean":"2023-09-12T14:54:59.806750+0000","last_became_active":"2023-09-12T14:54:58.727977+0000","last_became_peered":"2023-09-12T14:54:58.727977+0000","last_unstale":"2023-09-12T14:54:59.806750+0000","last_undegraded":"2023-09-12T14:54:59.806750+0000","last_fullsized":"2023-09-12T14:54:59.806750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:09:24.488408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.809447+0000","last_change":"2023-09-12T14:54:57.683980+0000","last_active":"2023-09-12T14:54:59.809447+0000","last_peered":"2023-09-12T14:54:59.809447+0000","last_clean":"2023-09-12T14:54:59.809447+0000","last_became_active":"2023-09-12T14:54:57.683572+0000","last_became_peered":"2023-09-12T14:54:57.683572+0000","last_unstale":"2023-09-12T14:54:59.809447+0000","last_undegraded":"2023-09-12T14:54:59.809447+0000","last_fullsized":"2023-09-12T14:54:59.809447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:10:06.188199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.809386+0000","last_change":"2023-09-12T14:54:58.758048+0000","last_active":"2023-09-12T14:54:59.809386+0000","last_peered":"2023-09-12T14:54:59.809386+0000","last_clean":"2023-09-12T14:54:59.809386+0000","last_became_active":"2023-09-12T14:54:58.757449+0000","last_became_peered":"2023-09-12T14:54:58.757449+0000","last_unstale":"2023-09-12T14:54:59.809386+0000","last_undegraded":"2023-09-12T14:54:59.809386+0000","last_fullsized":"2023-09-12T14:54:59.809386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:07:26.691746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807331+0000","last_change":"2023-09-12T14:54:57.709447+0000","last_active":"2023-09-12T14:54:59.807331+0000","last_peered":"2023-09-12T14:54:59.807331+0000","last_clean":"2023-09-12T14:54:59.807331+0000","last_became_active":"2023-09-12T14:54:57.709126+0000","last_became_peered":"2023-09-12T14:54:57.709126+0000","last_unstale":"2023-09-12T14:54:59.807331+0000","last_undegraded":"2023-09-12T14:54:59.807331+0000","last_fullsized":"2023-09-12T14:54:59.807331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:34:38.380062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.337}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":70988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":806912,"data_stored":660885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-12T14:55:02.351 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-12T14:55:02.352 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2023-09-12T14:55:02.352 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-09-12T14:55:02.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.513+0000 7faad28d1640 1 -- 172.21.15.131:0/3037501353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faacc0ab730 msgr2=0x7faacc0abb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:02.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.513+0000 7faad28d1640 1 --2- 172.21.15.131:0/3037501353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faacc0ab730 0x7faacc0abb10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7faacc06cd90 tx=0x7faabc01c870 comp rx=0 tx=0).stop 2023-09-12T14:55:02.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.513+0000 7faad28d1640 1 -- 172.21.15.131:0/3037501353 shutdown_connections 2023-09-12T14:55:02.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.513+0000 7faad28d1640 1 --2- 172.21.15.131:0/3037501353 >> v2:172.21.15.187:3300/0 conn(0x7faacc06f390 0x7faacc06f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:02.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.513+0000 7faad28d1640 1 --2- 172.21.15.131:0/3037501353 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faacc158180 0x7faacc0cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:02.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.513+0000 7faad28d1640 1 --2- 172.21.15.131:0/3037501353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faacc0ab730 0x7faacc0abb10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:02.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.513+0000 7faad28d1640 1 -- 172.21.15.131:0/3037501353 >> 172.21.15.131:0/3037501353 conn(0x7faacc069840 msgr2=0x7faacc0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:02.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.513+0000 7faad28d1640 1 -- 172.21.15.131:0/3037501353 shutdown_connections 2023-09-12T14:55:02.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.513+0000 7faad28d1640 1 -- 172.21.15.131:0/3037501353 wait complete. 2023-09-12T14:55:02.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.517+0000 7faad28d1640 1 Processor -- start 2023-09-12T14:55:02.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.517+0000 7faad28d1640 1 -- start start 2023-09-12T14:55:02.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.517+0000 7faad28d1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faacc06f390 0x7faacc1592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:02.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.517+0000 7faad28d1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faacc0ab730 0x7faacc14a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:02.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.517+0000 7faad28d1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faacc158180 0x7faacc156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:02.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.517+0000 7faad28d1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faacc150340 con 0x7faacc0ab730 2023-09-12T14:55:02.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.517+0000 7faad28d1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faacc159810 con 0x7faacc06f390 2023-09-12T14:55:02.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.517+0000 7faad28d1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faacc14f6c0 con 0x7faacc158180 2023-09-12T14:55:02.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.517+0000 7faac3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faacc0ab730 0x7faacc14a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:02.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.517+0000 7faac3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faacc0ab730 0x7faacc14a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38790/0 (socket says 172.21.15.131:38790) 2023-09-12T14:55:02.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.517+0000 7faac3fff640 1 -- 172.21.15.131:0/2603130512 learned_addr learned my addr 172.21.15.131:0/2603130512 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:02.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.517+0000 7faad0e6e640 1 --2- 172.21.15.131:0/2603130512 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faacc158180 0x7faacc156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:02.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.517+0000 7faacbfff640 1 --2- 172.21.15.131:0/2603130512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faacc06f390 0x7faacc1592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:02.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.517+0000 7faac3fff640 1 -- 172.21.15.131:0/2603130512 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faacc158180 msgr2=0x7faacc156b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:02.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.517+0000 7faac3fff640 1 --2- 172.21.15.131:0/2603130512 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faacc158180 0x7faacc156b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:02.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.517+0000 7faac3fff640 1 -- 172.21.15.131:0/2603130512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faacc06f390 msgr2=0x7faacc1592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:02.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.517+0000 7faac3fff640 1 --2- 172.21.15.131:0/2603130512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faacc06f390 0x7faacc1592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:02.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.517+0000 7faac3fff640 1 -- 172.21.15.131:0/2603130512 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faabc009560 con 0x7faacc0ab730 2023-09-12T14:55:02.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.517+0000 7faac3fff640 1 --2- 172.21.15.131:0/2603130512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faacc0ab730 0x7faacc14a1f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7faab400b410 tx=0x7faab400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:02.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.517+0000 7faac97fa640 1 -- 172.21.15.131:0/2603130512 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab4011830 con 0x7faacc0ab730 2023-09-12T14:55:02.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.517+0000 7faac97fa640 1 -- 172.21.15.131:0/2603130512 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faab4011a80 con 0x7faacc0ab730 2023-09-12T14:55:02.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.517+0000 7faad28d1640 1 -- 172.21.15.131:0/2603130512 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faacc14f9a0 con 0x7faacc0ab730 2023-09-12T14:55:02.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.517+0000 7faac97fa640 1 -- 172.21.15.131:0/2603130512 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab4011e90 con 0x7faacc0ab730 2023-09-12T14:55:02.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.517+0000 7faad28d1640 1 -- 172.21.15.131:0/2603130512 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faacc14ff60 con 0x7faacc0ab730 2023-09-12T14:55:02.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.517+0000 7faad28d1640 1 -- 172.21.15.131:0/2603130512 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faacc06f820 con 0x7faacc0ab730 2023-09-12T14:55:02.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.521+0000 7faac97fa640 1 -- 172.21.15.131:0/2603130512 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7faab401a400 con 0x7faacc0ab730 2023-09-12T14:55:02.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.521+0000 7faac97fa640 1 --2- 172.21.15.131:0/2603130512 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7faaac065b20 0x7faaac067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:02.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.521+0000 7faacbfff640 1 --2- 172.21.15.131:0/2603130512 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7faaac065b20 0x7faaac067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:02.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.521+0000 7faac97fa640 1 -- 172.21.15.131:0/2603130512 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faab40836b0 con 0x7faacc0ab730 2023-09-12T14:55:02.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.521+0000 7faacbfff640 1 --2- 172.21.15.131:0/2603130512 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7faaac065b20 0x7faaac067fe0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faabc0af000 tx=0x7faabc01b0b0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:02.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.525+0000 7faac97fa640 1 -- 172.21.15.131:0/2603130512 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7faab4051140 con 0x7faacc0ab730 2023-09-12T14:55:02.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.665+0000 7faad28d1640 1 -- 172.21.15.131:0/2603130512 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) v1 -- 0x7faacc152550 con 0x7faacc0ab730 2023-09-12T14:55:02.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.669+0000 7faac97fa640 1 -- 172.21.15.131:0/2603130512 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v30) v1 ==== 86+0+160 (secure 0 0 0) 0x7faab40150b0 con 0x7faacc0ab730 2023-09-12T14:55:02.670 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:02.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.669+0000 7faad28d1640 1 -- 172.21.15.131:0/2603130512 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7faaac065b20 msgr2=0x7faaac067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:02.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.669+0000 7faad28d1640 1 --2- 172.21.15.131:0/2603130512 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7faaac065b20 0x7faaac067fe0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faabc0af000 tx=0x7faabc01b0b0 comp rx=0 tx=0).stop 2023-09-12T14:55:02.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.669+0000 7faad28d1640 1 -- 172.21.15.131:0/2603130512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faacc0ab730 msgr2=0x7faacc14a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:02.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.669+0000 7faad28d1640 1 --2- 172.21.15.131:0/2603130512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faacc0ab730 0x7faacc14a1f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7faab400b410 tx=0x7faab400b8d0 comp rx=0 tx=0).stop 2023-09-12T14:55:02.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.669+0000 7faad28d1640 1 -- 172.21.15.131:0/2603130512 shutdown_connections 2023-09-12T14:55:02.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.669+0000 7faad28d1640 1 --2- 172.21.15.131:0/2603130512 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7faaac065b20 0x7faaac067fe0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:02.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.669+0000 7faad28d1640 1 --2- 172.21.15.131:0/2603130512 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faacc158180 0x7faacc156b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:02.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.669+0000 7faad28d1640 1 --2- 172.21.15.131:0/2603130512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faacc0ab730 0x7faacc14a1f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:02.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.669+0000 7faad28d1640 1 --2- 172.21.15.131:0/2603130512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faacc06f390 0x7faacc1592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:02.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.669+0000 7faad28d1640 1 -- 172.21.15.131:0/2603130512 >> 172.21.15.131:0/2603130512 conn(0x7faacc069840 msgr2=0x7faacc0ade90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:02.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.669+0000 7faad28d1640 1 -- 172.21.15.131:0/2603130512 shutdown_connections 2023-09-12T14:55:02.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.669+0000 7faad28d1640 1 -- 172.21.15.131:0/2603130512 wait complete. 2023-09-12T14:55:02.687 INFO:teuthology.orchestra.run.smithi131.stdout:[{"version":30,"timestamp":"2023-09-12T14:54:44.079877+0000","name":"","changes":[{"name":"osd.0/osd_mclock_max_capacity_iops_ssd","new_value":"3490.021839"}]}] 2023-09-12T14:55:02.687 INFO:tasks.ceph_manager:config epoch is 30 2023-09-12T14:55:02.688 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-09-12T14:55:02.688 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-09-12T14:55:02.688 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-09-12T14:55:02.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f714af6d640 1 -- 172.21.15.131:0/1431181654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f714406b050 msgr2=0x7f714406b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:02.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f714af6d640 1 --2- 172.21.15.131:0/1431181654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f714406b050 0x7f714406b460 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f714406eea0 tx=0x7f713c01ccf0 comp rx=0 tx=0).stop 2023-09-12T14:55:02.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f714af6d640 1 -- 172.21.15.131:0/1431181654 shutdown_connections 2023-09-12T14:55:02.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f714af6d640 1 --2- 172.21.15.131:0/1431181654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f714406b050 0x7f714406b460 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:02.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f714af6d640 1 --2- 172.21.15.131:0/1431181654 >> v2:172.21.15.187:3300/0 conn(0x7f71440ab730 0x7f71440abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:02.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f714af6d640 1 -- 172.21.15.131:0/1431181654 >> 172.21.15.131:0/1431181654 conn(0x7f7144069840 msgr2=0x7f71440cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:02.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f714af6d640 1 -- 172.21.15.131:0/1431181654 shutdown_connections 2023-09-12T14:55:02.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f714af6d640 1 -- 172.21.15.131:0/1431181654 wait complete. 2023-09-12T14:55:02.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f714af6d640 1 Processor -- start 2023-09-12T14:55:02.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f714af6d640 1 -- start start 2023-09-12T14:55:02.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f714af6d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f714406b050 0x7f7144157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:02.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f714af6d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71440ab730 0x7f7144153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:02.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f714af6d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7144158180 0x7f71440ad950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:02.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f714af6d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7144144b70 con 0x7f7144158180 2023-09-12T14:55:02.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f714af6d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7144144e50 con 0x7f714406b050 2023-09-12T14:55:02.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f714af6d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7144145130 con 0x7f71440ab730 2023-09-12T14:55:02.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f714950a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7144158180 0x7f71440ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:02.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f714950a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7144158180 0x7f71440ad950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38818/0 (socket says 172.21.15.131:38818) 2023-09-12T14:55:02.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f714950a640 1 -- 172.21.15.131:0/74486004 learned_addr learned my addr 172.21.15.131:0/74486004 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:02.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f713bfff640 1 --2- 172.21.15.131:0/74486004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71440ab730 0x7f7144153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:02.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f714950a640 1 -- 172.21.15.131:0/74486004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71440ab730 msgr2=0x7f7144153150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:02.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f7148d09640 1 --2- 172.21.15.131:0/74486004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f714406b050 0x7f7144157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:02.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f714950a640 1 --2- 172.21.15.131:0/74486004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71440ab730 0x7f7144153150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:02.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f714950a640 1 -- 172.21.15.131:0/74486004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f714406b050 msgr2=0x7f7144157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:02.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f714950a640 1 --2- 172.21.15.131:0/74486004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f714406b050 0x7f7144157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:02.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f714950a640 1 -- 172.21.15.131:0/74486004 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713c0098e0 con 0x7f7144158180 2023-09-12T14:55:02.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f713bfff640 1 --2- 172.21.15.131:0/74486004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71440ab730 0x7f7144153150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:02.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f7148d09640 1 --2- 172.21.15.131:0/74486004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f714406b050 0x7f7144157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:02.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f714950a640 1 --2- 172.21.15.131:0/74486004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7144158180 0x7f71440ad950 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f713c01a920 tx=0x7f713c0a5dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:02.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f71397fa640 1 -- 172.21.15.131:0/74486004 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c0adab0 con 0x7f7144158180 2023-09-12T14:55:02.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f71397fa640 1 -- 172.21.15.131:0/74486004 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f713c0add00 con 0x7f7144158180 2023-09-12T14:55:02.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f714af6d640 1 -- 172.21.15.131:0/74486004 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7144145410 con 0x7f7144158180 2023-09-12T14:55:02.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f71397fa640 1 -- 172.21.15.131:0/74486004 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c0041c0 con 0x7f7144158180 2023-09-12T14:55:02.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.809+0000 7f714af6d640 1 -- 172.21.15.131:0/74486004 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7144152c90 con 0x7f7144158180 2023-09-12T14:55:02.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.813+0000 7f714af6d640 1 -- 172.21.15.131:0/74486004 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7144073580 con 0x7f7144158180 2023-09-12T14:55:02.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.813+0000 7f71397fa640 1 -- 172.21.15.131:0/74486004 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f713c0b4040 con 0x7f7144158180 2023-09-12T14:55:02.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.813+0000 7f71397fa640 1 --2- 172.21.15.131:0/74486004 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7130065b50 0x7f7130068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:02.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.813+0000 7f71397fa640 1 -- 172.21.15.131:0/74486004 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f713c11e5f0 con 0x7f7144158180 2023-09-12T14:55:02.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.813+0000 7f7148d09640 1 --2- 172.21.15.131:0/74486004 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7130065b50 0x7f7130068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:02.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.813+0000 7f7148d09640 1 --2- 172.21.15.131:0/74486004 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7130065b50 0x7f7130068010 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f714000a8b0 tx=0x7f7140008040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:02.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:02.817+0000 7f71397fa640 1 -- 172.21.15.131:0/74486004 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f713c0ebed0 con 0x7f7144158180 2023-09-12T14:55:03.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.009+0000 7f714af6d640 1 -- 172.21.15.131:0/74486004 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f7144073580 con 0x7f7144158180 2023-09-12T14:55:03.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.013+0000 7f71397fa640 1 -- 172.21.15.131:0/74486004 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+163509 (secure 0 0 0) 0x7f713c0b2090 con 0x7f7144158180 2023-09-12T14:55:03.018 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:03.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.017+0000 7f714af6d640 1 -- 172.21.15.131:0/74486004 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7130065b50 msgr2=0x7f7130068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.017+0000 7f714af6d640 1 --2- 172.21.15.131:0/74486004 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7130065b50 0x7f7130068010 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f714000a8b0 tx=0x7f7140008040 comp rx=0 tx=0).stop 2023-09-12T14:55:03.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.017+0000 7f714af6d640 1 -- 172.21.15.131:0/74486004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7144158180 msgr2=0x7f71440ad950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.017+0000 7f714af6d640 1 --2- 172.21.15.131:0/74486004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7144158180 0x7f71440ad950 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f713c01a920 tx=0x7f713c0a5dc0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.017+0000 7f714af6d640 1 -- 172.21.15.131:0/74486004 shutdown_connections 2023-09-12T14:55:03.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.017+0000 7f714af6d640 1 --2- 172.21.15.131:0/74486004 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7130065b50 0x7f7130068010 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.017+0000 7f714af6d640 1 --2- 172.21.15.131:0/74486004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71440ab730 0x7f7144153150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.017+0000 7f714af6d640 1 --2- 172.21.15.131:0/74486004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7144158180 0x7f71440ad950 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.017+0000 7f714af6d640 1 --2- 172.21.15.131:0/74486004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f714406b050 0x7f7144157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.017+0000 7f714af6d640 1 -- 172.21.15.131:0/74486004 >> 172.21.15.131:0/74486004 conn(0x7f7144069840 msgr2=0x7f71441502b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:03.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.017+0000 7f714af6d640 1 -- 172.21.15.131:0/74486004 shutdown_connections 2023-09-12T14:55:03.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.017+0000 7f714af6d640 1 -- 172.21.15.131:0/74486004 wait complete. 2023-09-12T14:55:03.036 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":5,"active_gid":4200,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2298103651},{"type":"v1","addr":"172.21.15.131:6833","nonce":2298103651}]},"active_addr":"172.21.15.131:6833/2298103651","active_change":"2023-09-12T14:54:37.564401+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4175,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":2224553369}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":444139561}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":469229524}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":1463212059}]}]} 2023-09-12T14:55:03.040 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-09-12T14:55:03.041 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-09-12T14:55:03.041 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:55:03.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.165+0000 7f4c0432b640 1 -- 172.21.15.131:0/2400368585 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bfc0abae0 msgr2=0x7f4bfc0c9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.165+0000 7f4c0432b640 1 --2- 172.21.15.131:0/2400368585 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bfc0abae0 0x7f4bfc0c9310 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc071110 tx=0x7f4bec01c810 comp rx=0 tx=0).stop 2023-09-12T14:55:03.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.165+0000 7f4c0432b640 1 -- 172.21.15.131:0/2400368585 shutdown_connections 2023-09-12T14:55:03.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.165+0000 7f4c0432b640 1 --2- 172.21.15.131:0/2400368585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bfc0cc430 0x7f4bfc0c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.165+0000 7f4c0432b640 1 --2- 172.21.15.131:0/2400368585 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bfc0abae0 0x7f4bfc0c9310 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.165+0000 7f4c0432b640 1 --2- 172.21.15.131:0/2400368585 >> v2:172.21.15.187:3300/0 conn(0x7f4bfc158d70 0x7f4bfc06b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.165+0000 7f4c0432b640 1 -- 172.21.15.131:0/2400368585 >> 172.21.15.131:0/2400368585 conn(0x7f4bfc069840 msgr2=0x7f4bfc0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:03.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.165+0000 7f4c0432b640 1 -- 172.21.15.131:0/2400368585 shutdown_connections 2023-09-12T14:55:03.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.165+0000 7f4c0432b640 1 -- 172.21.15.131:0/2400368585 wait complete. 2023-09-12T14:55:03.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.165+0000 7f4c0432b640 1 Processor -- start 2023-09-12T14:55:03.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.165+0000 7f4c0432b640 1 -- start start 2023-09-12T14:55:03.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.169+0000 7f4c0432b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bfc0abae0 0x7f4bfc149ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.169+0000 7f4c0432b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bfc0cc430 0x7f4bfc1514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.169+0000 7f4c0432b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4bfc158d70 0x7f4bfc0aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.169+0000 7f4c0432b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc15bca0 con 0x7f4bfc0abae0 2023-09-12T14:55:03.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.169+0000 7f4c0432b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc15b090 con 0x7f4bfc158d70 2023-09-12T14:55:03.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.169+0000 7f4c0432b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc15b370 con 0x7f4bfc0cc430 2023-09-12T14:55:03.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.169+0000 7f4c020c7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bfc0abae0 0x7f4bfc149ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.169+0000 7f4c020c7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bfc0abae0 0x7f4bfc149ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38834/0 (socket says 172.21.15.131:38834) 2023-09-12T14:55:03.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.169+0000 7f4c020c7640 1 -- 172.21.15.131:0/1739245153 learned_addr learned my addr 172.21.15.131:0/1739245153 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:03.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.169+0000 7f4c020c7640 1 -- 172.21.15.131:0/1739245153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bfc0cc430 msgr2=0x7f4bfc1514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.169+0000 7f4c020c7640 1 --2- 172.21.15.131:0/1739245153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bfc0cc430 0x7f4bfc1514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.169+0000 7f4c020c7640 1 -- 172.21.15.131:0/1739245153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4bfc158d70 msgr2=0x7f4bfc0aa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.169+0000 7f4c020c7640 1 --2- 172.21.15.131:0/1739245153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4bfc158d70 0x7f4bfc0aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.169+0000 7f4c020c7640 1 -- 172.21.15.131:0/1739245153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bec009560 con 0x7f4bfc0abae0 2023-09-12T14:55:03.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.169+0000 7f4c020c7640 1 --2- 172.21.15.131:0/1739245153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bfc0abae0 0x7f4bfc149ff0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800b410 tx=0x7f4bf800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.169+0000 7f4bf2ffd640 1 -- 172.21.15.131:0/1739245153 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8011830 con 0x7f4bfc0abae0 2023-09-12T14:55:03.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.169+0000 7f4bf2ffd640 1 -- 172.21.15.131:0/1739245153 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4bf8011aa0 con 0x7f4bfc0abae0 2023-09-12T14:55:03.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.169+0000 7f4c0432b640 1 -- 172.21.15.131:0/1739245153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bfc15b650 con 0x7f4bfc0abae0 2023-09-12T14:55:03.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.169+0000 7f4bf2ffd640 1 -- 172.21.15.131:0/1739245153 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8010380 con 0x7f4bfc0abae0 2023-09-12T14:55:03.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.169+0000 7f4c0432b640 1 -- 172.21.15.131:0/1739245153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bfc14aa70 con 0x7f4bfc0abae0 2023-09-12T14:55:03.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.169+0000 7f4c0432b640 1 -- 172.21.15.131:0/1739245153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bfc073580 con 0x7f4bfc0abae0 2023-09-12T14:55:03.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.169+0000 7f4bf2ffd640 1 -- 172.21.15.131:0/1739245153 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f4bf80104e0 con 0x7f4bfc0abae0 2023-09-12T14:55:03.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.169+0000 7f4bf2ffd640 1 --2- 172.21.15.131:0/1739245153 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4be0065b20 0x7f4be0067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.169+0000 7f4c018c6640 1 --2- 172.21.15.131:0/1739245153 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4be0065b20 0x7f4be0067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.169+0000 7f4bf2ffd640 1 -- 172.21.15.131:0/1739245153 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4bf80836b0 con 0x7f4bfc0abae0 2023-09-12T14:55:03.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.169+0000 7f4c018c6640 1 --2- 172.21.15.131:0/1739245153 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4be0065b20 0x7f4be0067fe0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0ae000 tx=0x7f4bec0195b0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.173+0000 7f4bf2ffd640 1 -- 172.21.15.131:0/1739245153 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f4bf8051090 con 0x7f4bfc0abae0 2023-09-12T14:55:03.319 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.317+0000 7f4c0432b640 1 -- 172.21.15.131:0/1739245153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4bfc0cd4d0 con 0x7f4bfc0abae0 2023-09-12T14:55:03.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.317+0000 7f4bf2ffd640 1 -- 172.21.15.131:0/1739245153 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f4bf80546b0 con 0x7f4bfc0abae0 2023-09-12T14:55:03.321 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:03.321 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:54:59.694778+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:54:55.689983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T14:54:57.029447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:55:03.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.321+0000 7f4c0432b640 1 -- 172.21.15.131:0/1739245153 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4be0065b20 msgr2=0x7f4be0067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.321+0000 7f4c0432b640 1 --2- 172.21.15.131:0/1739245153 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4be0065b20 0x7f4be0067fe0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0ae000 tx=0x7f4bec0195b0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.321+0000 7f4c0432b640 1 -- 172.21.15.131:0/1739245153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bfc0abae0 msgr2=0x7f4bfc149ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.321+0000 7f4c0432b640 1 --2- 172.21.15.131:0/1739245153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bfc0abae0 0x7f4bfc149ff0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800b410 tx=0x7f4bf800b8d0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.321+0000 7f4c0432b640 1 -- 172.21.15.131:0/1739245153 shutdown_connections 2023-09-12T14:55:03.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.321+0000 7f4c0432b640 1 --2- 172.21.15.131:0/1739245153 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4be0065b20 0x7f4be0067fe0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.321+0000 7f4c0432b640 1 --2- 172.21.15.131:0/1739245153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bfc0cc430 0x7f4bfc1514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.321+0000 7f4c0432b640 1 --2- 172.21.15.131:0/1739245153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4bfc158d70 0x7f4bfc0aa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.321+0000 7f4c0432b640 1 --2- 172.21.15.131:0/1739245153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bfc0abae0 0x7f4bfc149ff0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.321+0000 7f4c0432b640 1 -- 172.21.15.131:0/1739245153 >> 172.21.15.131:0/1739245153 conn(0x7f4bfc069840 msgr2=0x7f4bfc14f650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:03.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.321+0000 7f4c0432b640 1 -- 172.21.15.131:0/1739245153 shutdown_connections 2023-09-12T14:55:03.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.321+0000 7f4c0432b640 1 -- 172.21.15.131:0/1739245153 wait complete. 2023-09-12T14:55:03.338 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-09-12T14:55:03.338 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:55:03.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16fa87d640 1 -- 172.21.15.131:0/2578816392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16f4143be0 msgr2=0x7f16f415e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16fa87d640 1 --2- 172.21.15.131:0/2578816392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16f4143be0 0x7f16f415e960 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f16ec009bc0 tx=0x7f16ec01cb50 comp rx=0 tx=0).stop 2023-09-12T14:55:03.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16fa87d640 1 -- 172.21.15.131:0/2578816392 shutdown_connections 2023-09-12T14:55:03.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16fa87d640 1 --2- 172.21.15.131:0/2578816392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16f4143be0 0x7f16f415e960 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16fa87d640 1 --2- 172.21.15.131:0/2578816392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16f4143800 0x7f16f406b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16fa87d640 1 -- 172.21.15.131:0/2578816392 >> 172.21.15.131:0/2578816392 conn(0x7f16f4069840 msgr2=0x7f16f40cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:03.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16fa87d640 1 -- 172.21.15.131:0/2578816392 shutdown_connections 2023-09-12T14:55:03.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16fa87d640 1 -- 172.21.15.131:0/2578816392 wait complete. 2023-09-12T14:55:03.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16fa87d640 1 Processor -- start 2023-09-12T14:55:03.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16fa87d640 1 -- start start 2023-09-12T14:55:03.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16fa87d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16f4143800 0x7f16f4070060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16fa87d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16f4143be0 0x7f16f40ac790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16fa87d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16f41440e0 0x7f16f415a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16fa87d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f40b1c70 con 0x7f16f41440e0 2023-09-12T14:55:03.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16fa87d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f40b2100 con 0x7f16f4143800 2023-09-12T14:55:03.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16fa87d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16f40b1060 con 0x7f16f4143be0 2023-09-12T14:55:03.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16f8e1a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16f41440e0 0x7f16f415a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16f8e1a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16f41440e0 0x7f16f415a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38850/0 (socket says 172.21.15.131:38850) 2023-09-12T14:55:03.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16f8e1a640 1 -- 172.21.15.131:0/3156276665 learned_addr learned my addr 172.21.15.131:0/3156276665 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:03.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16f8e1a640 1 -- 172.21.15.131:0/3156276665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16f4143be0 msgr2=0x7f16f40ac790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16f8e1a640 1 --2- 172.21.15.131:0/3156276665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16f4143be0 0x7f16f40ac790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16f8e1a640 1 -- 172.21.15.131:0/3156276665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16f4143800 msgr2=0x7f16f4070060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16f8e1a640 1 --2- 172.21.15.131:0/3156276665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16f4143800 0x7f16f4070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16f8e1a640 1 -- 172.21.15.131:0/3156276665 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ec009820 con 0x7f16f41440e0 2023-09-12T14:55:03.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16f8e1a640 1 --2- 172.21.15.131:0/3156276665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16f41440e0 0x7f16f415a190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f16ec009b10 tx=0x7f16ec0a5ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16f0ff9640 1 -- 172.21.15.131:0/3156276665 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec0041e0 con 0x7f16f41440e0 2023-09-12T14:55:03.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16f0ff9640 1 -- 172.21.15.131:0/3156276665 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f16ec004430 con 0x7f16f41440e0 2023-09-12T14:55:03.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16fa87d640 1 -- 172.21.15.131:0/3156276665 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f40b1340 con 0x7f16f41440e0 2023-09-12T14:55:03.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.449+0000 7f16f0ff9640 1 -- 172.21.15.131:0/3156276665 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec0047a0 con 0x7f16f41440e0 2023-09-12T14:55:03.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.453+0000 7f16fa87d640 1 -- 172.21.15.131:0/3156276665 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f40b1900 con 0x7f16f41440e0 2023-09-12T14:55:03.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.453+0000 7f16fa87d640 1 -- 172.21.15.131:0/3156276665 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f4073580 con 0x7f16f41440e0 2023-09-12T14:55:03.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.453+0000 7f16f0ff9640 1 -- 172.21.15.131:0/3156276665 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f16ec0b4040 con 0x7f16f41440e0 2023-09-12T14:55:03.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.453+0000 7f16f0ff9640 1 --2- 172.21.15.131:0/3156276665 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f16e0065b50 0x7f16e0068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.453+0000 7f16f0ff9640 1 -- 172.21.15.131:0/3156276665 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16ec11e140 con 0x7f16f41440e0 2023-09-12T14:55:03.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.453+0000 7f16f3fff640 1 --2- 172.21.15.131:0/3156276665 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f16e0065b50 0x7f16e0068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.453+0000 7f16f3fff640 1 --2- 172.21.15.131:0/3156276665 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f16e0065b50 0x7f16e0068010 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f16e400a910 tx=0x7f16e4005c00 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.457+0000 7f16f0ff9640 1 -- 172.21.15.131:0/3156276665 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f16ec0eba90 con 0x7f16f41440e0 2023-09-12T14:55:03.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.601+0000 7f16fa87d640 1 -- 172.21.15.131:0/3156276665 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16f40cd790 con 0x7f16f41440e0 2023-09-12T14:55:03.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.601+0000 7f16f0ff9640 1 -- 172.21.15.131:0/3156276665 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f16ec0ef0b0 con 0x7f16f41440e0 2023-09-12T14:55:03.605 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:03.606 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:54:59.694778+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:54:55.689983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T14:54:57.029447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:55:03.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.605+0000 7f16fa87d640 1 -- 172.21.15.131:0/3156276665 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f16e0065b50 msgr2=0x7f16e0068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.605+0000 7f16fa87d640 1 --2- 172.21.15.131:0/3156276665 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f16e0065b50 0x7f16e0068010 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f16e400a910 tx=0x7f16e4005c00 comp rx=0 tx=0).stop 2023-09-12T14:55:03.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.605+0000 7f16fa87d640 1 -- 172.21.15.131:0/3156276665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16f41440e0 msgr2=0x7f16f415a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.605+0000 7f16fa87d640 1 --2- 172.21.15.131:0/3156276665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16f41440e0 0x7f16f415a190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f16ec009b10 tx=0x7f16ec0a5ba0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.605+0000 7f16fa87d640 1 -- 172.21.15.131:0/3156276665 shutdown_connections 2023-09-12T14:55:03.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.605+0000 7f16fa87d640 1 --2- 172.21.15.131:0/3156276665 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f16e0065b50 0x7f16e0068010 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.605+0000 7f16fa87d640 1 --2- 172.21.15.131:0/3156276665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16f4143be0 0x7f16f40ac790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.605+0000 7f16fa87d640 1 --2- 172.21.15.131:0/3156276665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16f41440e0 0x7f16f415a190 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.605+0000 7f16fa87d640 1 --2- 172.21.15.131:0/3156276665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16f4143800 0x7f16f4070060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.605+0000 7f16fa87d640 1 -- 172.21.15.131:0/3156276665 >> 172.21.15.131:0/3156276665 conn(0x7f16f4069840 msgr2=0x7f16f40ad930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:03.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.605+0000 7f16fa87d640 1 -- 172.21.15.131:0/3156276665 shutdown_connections 2023-09-12T14:55:03.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.605+0000 7f16fa87d640 1 -- 172.21.15.131:0/3156276665 wait complete. 2023-09-12T14:55:03.623 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-09-12T14:55:03.624 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-09-12T14:55:03.624 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-09-12T14:55:03.625 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-09-12T14:55:03.625 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-09-12T14:55:03.626 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-09-12T14:55:03.627 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-09-12T14:55:03.627 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-09-12T14:55:03.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.809+0000 7fe9c5590640 1 -- 172.21.15.131:0/3611779582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9c00ccc50 msgr2=0x7fe9c01576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.809+0000 7fe9c5590640 1 --2- 172.21.15.131:0/3611779582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9c00ccc50 0x7fe9c01576a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00afdc0 tx=0x7fe9b401db10 comp rx=0 tx=0).stop 2023-09-12T14:55:03.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.809+0000 7fe9c5590640 1 -- 172.21.15.131:0/3611779582 shutdown_connections 2023-09-12T14:55:03.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.809+0000 7fe9c5590640 1 --2- 172.21.15.131:0/3611779582 >> v2:172.21.15.187:3300/0 conn(0x7fe9c00b0b80 0x7fe9c0160d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.809+0000 7fe9c5590640 1 --2- 172.21.15.131:0/3611779582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9c00ccc50 0x7fe9c01576a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.809+0000 7fe9c5590640 1 -- 172.21.15.131:0/3611779582 >> 172.21.15.131:0/3611779582 conn(0x7fe9c006b3c0 msgr2=0x7fe9c006b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:03.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.809+0000 7fe9c5590640 1 -- 172.21.15.131:0/3611779582 shutdown_connections 2023-09-12T14:55:03.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.809+0000 7fe9c5590640 1 -- 172.21.15.131:0/3611779582 wait complete. 2023-09-12T14:55:03.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.809+0000 7f6c7c98c640 1 -- 172.21.15.131:0/3518008257 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c780c8ec0 msgr2=0x7f6c780c9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.809+0000 7f6c7c98c640 1 --2- 172.21.15.131:0/3518008257 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c780c8ec0 0x7f6c780c9350 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6c68009930 tx=0x7f6c6801c820 comp rx=0 tx=0).stop 2023-09-12T14:55:03.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.809+0000 7f6c7c98c640 1 -- 172.21.15.131:0/3518008257 shutdown_connections 2023-09-12T14:55:03.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.809+0000 7f6c7c98c640 1 --2- 172.21.15.131:0/3518008257 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c780c8ec0 0x7f6c780c9350 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.809+0000 7f6c7c98c640 1 -- 172.21.15.131:0/3518008257 >> 172.21.15.131:0/3518008257 conn(0x7f6c780ce870 msgr2=0x7f6c780cec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:03.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.809+0000 7f6c7c98c640 1 -- 172.21.15.131:0/3518008257 shutdown_connections 2023-09-12T14:55:03.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.809+0000 7f6c7c98c640 1 -- 172.21.15.131:0/3518008257 wait complete. 2023-09-12T14:55:03.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6465a33640 1 -- 172.21.15.131:0/3846979236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6460156e80 msgr2=0x7f646006eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6465a33640 1 --2- 172.21.15.131:0/3846979236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6460156e80 0x7f646006eca0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6450009840 tx=0x7f645001c800 comp rx=0 tx=0).stop 2023-09-12T14:55:03.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6c7c98c640 1 Processor -- start 2023-09-12T14:55:03.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe806ffd640 1 -- 172.21.15.131:0/2265601010 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8000ad660 con 0x7fe8080abc20 2023-09-12T14:55:03.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe80e47b640 1 -- 172.21.15.131:0/2265601010 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8080abc20 msgr2=0x7fe80806ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe80e47b640 1 --2- 172.21.15.131:0/2265601010 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8080abc20 0x7fe80806ca30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe800009bc0 tx=0x7fe80001cad0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe80e47b640 1 -- 172.21.15.131:0/2265601010 shutdown_connections 2023-09-12T14:55:03.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe80e47b640 1 --2- 172.21.15.131:0/2265601010 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8080abc20 0x7fe80806ca30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe80e47b640 1 -- 172.21.15.131:0/2265601010 >> 172.21.15.131:0/2265601010 conn(0x7fe80806b470 msgr2=0x7fe80806b880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:03.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe80e47b640 1 -- 172.21.15.131:0/2265601010 shutdown_connections 2023-09-12T14:55:03.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6465a33640 1 -- 172.21.15.131:0/3846979236 shutdown_connections 2023-09-12T14:55:03.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6465a33640 1 --2- 172.21.15.131:0/3846979236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6460156e80 0x7f646006eca0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6465a33640 1 -- 172.21.15.131:0/3846979236 >> 172.21.15.131:0/3846979236 conn(0x7f6460069840 msgr2=0x7f64600cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:03.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6465a33640 1 -- 172.21.15.131:0/3846979236 shutdown_connections 2023-09-12T14:55:03.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6465a33640 1 -- 172.21.15.131:0/3846979236 wait complete. 2023-09-12T14:55:03.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6465a33640 1 Processor -- start 2023-09-12T14:55:03.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6c7c98c640 1 -- start start 2023-09-12T14:55:03.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6c7c98c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6c780c8ec0 0x7f6c78153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6c7c98c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c780cb350 0x7f6c7806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6c7c98c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c780cb730 0x7f6c78159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6c7c98c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c78153770 con 0x7f6c780c8ec0 2023-09-12T14:55:03.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6c7c98c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c78159250 con 0x7f6c780cb730 2023-09-12T14:55:03.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6c7c98c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c78153980 con 0x7f6c780cb350 2023-09-12T14:55:03.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6c7759e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c780cb730 0x7f6c78159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6c7759e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c780cb730 0x7f6c78159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:53758/0 (socket says 172.21.15.131:53758) 2023-09-12T14:55:03.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6c7759e640 1 -- 172.21.15.131:0/4182048866 learned_addr learned my addr 172.21.15.131:0/4182048866 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:03.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6c6ffff640 1 --2- 172.21.15.131:0/4182048866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c780cb350 0x7f6c7806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6c6ffff640 1 -- 172.21.15.131:0/4182048866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c780cb730 msgr2=0x7f6c78159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6c6ffff640 1 --2- 172.21.15.131:0/4182048866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c780cb730 0x7f6c78159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6c6ffff640 1 -- 172.21.15.131:0/4182048866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6c780c8ec0 msgr2=0x7f6c78153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6c6ffff640 1 --2- 172.21.15.131:0/4182048866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6c780c8ec0 0x7f6c78153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6c6ffff640 1 -- 172.21.15.131:0/4182048866 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c68009560 con 0x7f6c780cb350 2023-09-12T14:55:03.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6c6ffff640 1 --2- 172.21.15.131:0/4182048866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c780cb350 0x7f6c7806d3e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6c680ac000 tx=0x7f6c680a57d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f71b359e640 1 -- 172.21.15.131:0/3695913060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71b41406e0 msgr2=0x7f71b40c3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f71b359e640 1 --2- 172.21.15.131:0/3695913060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71b41406e0 0x7f71b40c3390 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f71b40cca60 tx=0x7f71a401c870 comp rx=0 tx=0).stop 2023-09-12T14:55:03.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe9c5590640 1 Processor -- start 2023-09-12T14:55:03.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe9c5590640 1 -- start start 2023-09-12T14:55:03.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe9c5590640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9c006fc20 0x7fe9c0156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe9c5590640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9c00b0b80 0x7fe9c01583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe9c5590640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9c00ccc50 0x7fe9c0145450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe9c5590640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c0159840 con 0x7fe9c00b0b80 2023-09-12T14:55:03.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe9c5590640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c0158920 con 0x7fe9c006fc20 2023-09-12T14:55:03.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe9c5590640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c0145990 con 0x7fe9c00ccc50 2023-09-12T14:55:03.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe9bf7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9c00ccc50 0x7fe9c0145450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe9bf7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9c00ccc50 0x7fe9c0145450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:42544/0 (socket says 172.21.15.131:42544) 2023-09-12T14:55:03.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe9bf7fe640 1 -- 172.21.15.131:0/1047406089 learned_addr learned my addr 172.21.15.131:0/1047406089 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:03.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe9beffd640 1 --2- 172.21.15.131:0/1047406089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9c006fc20 0x7fe9c0156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe80e47b640 1 -- 172.21.15.131:0/2265601010 wait complete. 2023-09-12T14:55:03.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe80e47b640 1 Processor -- start 2023-09-12T14:55:03.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f71b359e640 1 -- 172.21.15.131:0/3695913060 shutdown_connections 2023-09-12T14:55:03.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f71b359e640 1 --2- 172.21.15.131:0/3695913060 >> v2:172.21.15.187:3300/0 conn(0x7f71b415a830 0x7f71b415acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f71b359e640 1 --2- 172.21.15.131:0/3695913060 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71b40c9390 0x7f71b40c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f71b359e640 1 --2- 172.21.15.131:0/3695913060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71b41406e0 0x7f71b40c3390 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f71b359e640 1 -- 172.21.15.131:0/3695913060 >> 172.21.15.131:0/3695913060 conn(0x7f71b4069240 msgr2=0x7f71b4069650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:03.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6c6f7fe640 1 -- 172.21.15.131:0/4182048866 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c680ba070 con 0x7f6c780cb350 2023-09-12T14:55:03.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6c7c98c640 1 -- 172.21.15.131:0/4182048866 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c78153c60 con 0x7f6c780cb350 2023-09-12T14:55:03.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6c7c98c640 1 -- 172.21.15.131:0/4182048866 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c78159780 con 0x7f6c780cb350 2023-09-12T14:55:03.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f71b359e640 1 -- 172.21.15.131:0/3695913060 shutdown_connections 2023-09-12T14:55:03.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6c6f7fe640 1 -- 172.21.15.131:0/4182048866 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6c68003600 con 0x7f6c780cb350 2023-09-12T14:55:03.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6c6f7fe640 1 -- 172.21.15.131:0/4182048866 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c68003a10 con 0x7f6c780cb350 2023-09-12T14:55:03.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe9bf7fe640 1 -- 172.21.15.131:0/1047406089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9c006fc20 msgr2=0x7fe9c0156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe9bf7fe640 1 --2- 172.21.15.131:0/1047406089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9c006fc20 0x7fe9c0156790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe9bf7fe640 1 -- 172.21.15.131:0/1047406089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9c00b0b80 msgr2=0x7fe9c01583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe9bf7fe640 1 --2- 172.21.15.131:0/1047406089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9c00b0b80 0x7fe9c01583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe9bf7fe640 1 -- 172.21.15.131:0/1047406089 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b400a670 con 0x7fe9c00ccc50 2023-09-12T14:55:03.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe9bf7fe640 1 --2- 172.21.15.131:0/1047406089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9c00ccc50 0x7fe9c0145450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800eaf0 tx=0x7fe9b800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7f6465a33640 1 -- start start 2023-09-12T14:55:03.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f6465a33640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f646006baa0 0x7f646014d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f6465a33640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f646006f1e0 0x7f64601677a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f6465a33640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6460156e80 0x7f6460167ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f6465a33640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646014dd60 con 0x7f646006f1e0 2023-09-12T14:55:03.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f6465a33640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646014efe0 con 0x7f6460156e80 2023-09-12T14:55:03.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f6465a33640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6460151fd0 con 0x7f646006baa0 2023-09-12T14:55:03.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.813+0000 7fe80e47b640 1 -- start start 2023-09-12T14:55:03.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f645f7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6460156e80 0x7f6460167ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe80e47b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe80806cf70 0x7fe80815d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe80e47b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8080abc20 0x7fe8080c0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe80e47b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe80815e240 0x7fe8080afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe80e47b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8080c16c0 con 0x7fe8080abc20 2023-09-12T14:55:03.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe80e47b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8080c19a0 con 0x7fe80815e240 2023-09-12T14:55:03.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe80e47b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8080c1c80 con 0x7fe80806cf70 2023-09-12T14:55:03.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f645f7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6460156e80 0x7f6460167ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:53772/0 (socket says 172.21.15.131:53772) 2023-09-12T14:55:03.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f645f7fe640 1 -- 172.21.15.131:0/3830518603 learned_addr learned my addr 172.21.15.131:0/3830518603 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:03.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f645effd640 1 --2- 172.21.15.131:0/3830518603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f646006baa0 0x7f646014d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f64567fc640 1 --2- 172.21.15.131:0/3830518603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f646006f1e0 0x7f64601677a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe807fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe80806cf70 0x7fe80815d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe807fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe80806cf70 0x7fe80815d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:42594/0 (socket says 172.21.15.131:42594) 2023-09-12T14:55:03.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe807fff640 1 -- 172.21.15.131:0/1176752724 learned_addr learned my addr 172.21.15.131:0/1176752724 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:03.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe80ca18640 1 --2- 172.21.15.131:0/1176752724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe80815e240 0x7fe8080afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fac21862640 1 -- 172.21.15.131:0/1488499892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac1c0ac7c0 msgr2=0x7fac1c0cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fac21862640 1 --2- 172.21.15.131:0/1488499892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac1c0ac7c0 0x7fac1c0cba60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fac0c009900 tx=0x7fac0c01eff0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe9a77fe640 1 -- 172.21.15.131:0/1047406089 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8004b30 con 0x7fe9c00ccc50 2023-09-12T14:55:03.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe9a77fe640 1 -- 172.21.15.131:0/1047406089 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe9b8004cb0 con 0x7fe9c00ccc50 2023-09-12T14:55:03.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe9a77fe640 1 -- 172.21.15.131:0/1047406089 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8005280 con 0x7fe9c00ccc50 2023-09-12T14:55:03.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe9c5590640 1 -- 172.21.15.131:0/1047406089 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c0158bc0 con 0x7fe9c00ccc50 2023-09-12T14:55:03.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe9c5590640 1 -- 172.21.15.131:0/1047406089 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c0159180 con 0x7fe9c00ccc50 2023-09-12T14:55:03.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f645f7fe640 1 -- 172.21.15.131:0/3830518603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f646006baa0 msgr2=0x7f646014d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f645f7fe640 1 --2- 172.21.15.131:0/3830518603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f646006baa0 0x7f646014d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f645f7fe640 1 -- 172.21.15.131:0/3830518603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f646006f1e0 msgr2=0x7f64601677a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f645f7fe640 1 --2- 172.21.15.131:0/3830518603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f646006f1e0 0x7f64601677a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f645f7fe640 1 -- 172.21.15.131:0/3830518603 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64500094a0 con 0x7f6460156e80 2023-09-12T14:55:03.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fac21862640 1 -- 172.21.15.131:0/1488499892 shutdown_connections 2023-09-12T14:55:03.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fac21862640 1 --2- 172.21.15.131:0/1488499892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac1c0c7eb0 0x7fac1c06fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fac21862640 1 --2- 172.21.15.131:0/1488499892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac1c0ac7c0 0x7fac1c0cba60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fac21862640 1 -- 172.21.15.131:0/1488499892 >> 172.21.15.131:0/1488499892 conn(0x7fac1c0cd150 msgr2=0x7fac1c0cd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:03.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fac21862640 1 -- 172.21.15.131:0/1488499892 shutdown_connections 2023-09-12T14:55:03.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f71b359e640 1 -- 172.21.15.131:0/3695913060 wait complete. 2023-09-12T14:55:03.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f71b359e640 1 Processor -- start 2023-09-12T14:55:03.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe807fff640 1 -- 172.21.15.131:0/1176752724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe80815e240 msgr2=0x7fe8080afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe807fff640 1 --2- 172.21.15.131:0/1176752724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe80815e240 0x7fe8080afd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe807fff640 1 -- 172.21.15.131:0/1176752724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8080abc20 msgr2=0x7fe8080c0c20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-12T14:55:03.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe807fff640 1 --2- 172.21.15.131:0/1176752724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8080abc20 0x7fe8080c0c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe807fff640 1 -- 172.21.15.131:0/1176752724 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe800009820 con 0x7fe80806cf70 2023-09-12T14:55:03.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f645f7fe640 1 --2- 172.21.15.131:0/3830518603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6460156e80 0x7f6460167ce0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f645800cb70 tx=0x7f645800cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe807fff640 1 --2- 172.21.15.131:0/1176752724 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe80806cf70 0x7fe80815d680 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe7f800c410 tx=0x7fe7f800c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f645cff9640 1 -- 172.21.15.131:0/3830518603 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645800aa50 con 0x7f6460156e80 2023-09-12T14:55:03.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fef0dbe0640 1 -- 172.21.15.131:0/2348644559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef080afc20 msgr2=0x7fef080b4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fef0dbe0640 1 --2- 172.21.15.131:0/2348644559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef080afc20 0x7fef080b4b10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fef080c4470 tx=0x7fef0001ccf0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f6465a33640 1 -- 172.21.15.131:0/3830518603 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64601522b0 con 0x7f6460156e80 2023-09-12T14:55:03.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f6465a33640 1 -- 172.21.15.131:0/3830518603 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f646014f580 con 0x7f6460156e80 2023-09-12T14:55:03.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f645cff9640 1 -- 172.21.15.131:0/3830518603 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f645800aca0 con 0x7f6460156e80 2023-09-12T14:55:03.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f645cff9640 1 -- 172.21.15.131:0/3830518603 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6458012680 con 0x7f6460156e80 2023-09-12T14:55:03.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fac21862640 1 -- 172.21.15.131:0/1488499892 wait complete. 2023-09-12T14:55:03.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fac21862640 1 Processor -- start 2023-09-12T14:55:03.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f6c6f7fe640 1 -- 172.21.15.131:0/4182048866 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f6c680b4040 con 0x7f6c780cb350 2023-09-12T14:55:03.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f6c6f7fe640 1 --2- 172.21.15.131:0/4182048866 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6c58065b20 0x7f6c58067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f6c6f7fe640 1 -- 172.21.15.131:0/4182048866 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c6811da30 con 0x7f6c780cb350 2023-09-12T14:55:03.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f6c76d9d640 1 --2- 172.21.15.131:0/4182048866 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6c58065b20 0x7f6c58067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f71b359e640 1 -- start start 2023-09-12T14:55:03.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f71b359e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71b40c9390 0x7f71b40c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f71b359e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71b41406e0 0x7f71b413a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f71b359e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71b415a830 0x7f71b413e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f71b359e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b40b26d0 con 0x7f71b415a830 2023-09-12T14:55:03.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f71b359e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b40b2110 con 0x7f71b41406e0 2023-09-12T14:55:03.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f71b359e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b40b23f0 con 0x7f71b40c9390 2023-09-12T14:55:03.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe9c5590640 1 -- 172.21.15.131:0/1047406089 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe9c0073580 con 0x7fe9c00ccc50 2023-09-12T14:55:03.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe804ff9640 1 -- 172.21.15.131:0/1176752724 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8011790 con 0x7fe80806cf70 2023-09-12T14:55:03.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe804ff9640 1 -- 172.21.15.131:0/1176752724 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe7f80119e0 con 0x7fe80806cf70 2023-09-12T14:55:03.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe80e47b640 1 -- 172.21.15.131:0/1176752724 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8080c1f60 con 0x7fe80806cf70 2023-09-12T14:55:03.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe80e47b640 1 -- 172.21.15.131:0/1176752724 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8080b2180 con 0x7fe80806cf70 2023-09-12T14:55:03.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe804ff9640 1 -- 172.21.15.131:0/1176752724 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8011dd0 con 0x7fe80806cf70 2023-09-12T14:55:03.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fe80e47b640 1 -- 172.21.15.131:0/1176752724 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe80804e110 con 0x7fe80806cf70 2023-09-12T14:55:03.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fef0dbe0640 1 -- 172.21.15.131:0/2348644559 shutdown_connections 2023-09-12T14:55:03.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fef0dbe0640 1 --2- 172.21.15.131:0/2348644559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef080afc20 0x7fef080b4b10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fef0dbe0640 1 --2- 172.21.15.131:0/2348644559 >> v2:172.21.15.187:3300/0 conn(0x7fef080ccbc0 0x7fef080d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f71b259c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71b40c9390 0x7f71b40c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f71b259c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71b40c9390 0x7f71b40c6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:42606/0 (socket says 172.21.15.131:42606) 2023-09-12T14:55:03.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7f71b259c640 1 -- 172.21.15.131:0/3138093374 learned_addr learned my addr 172.21.15.131:0/3138093374 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:03.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f6c76d9d640 1 --2- 172.21.15.131:0/4182048866 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6c58065b20 0x7f6c58067fe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6c60003cf0 tx=0x7f6c600079c0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f71b1d9b640 1 --2- 172.21.15.131:0/3138093374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71b41406e0 0x7f71b413a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.817+0000 7fac21862640 1 -- start start 2023-09-12T14:55:03.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fac21862640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac1c06a640 0x7fac1c0c3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f71b259c640 1 -- 172.21.15.131:0/3138093374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71b41406e0 msgr2=0x7f71b413a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f71b259c640 1 --2- 172.21.15.131:0/3138093374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71b41406e0 0x7f71b413a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f71b259c640 1 -- 172.21.15.131:0/3138093374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71b415a830 msgr2=0x7f71b413e4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-12T14:55:03.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f71b259c640 1 --2- 172.21.15.131:0/3138093374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71b415a830 0x7f71b413e4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f71b259c640 1 -- 172.21.15.131:0/3138093374 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a4009560 con 0x7f71b40c9390 2023-09-12T14:55:03.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fac21862640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac1c0ac7c0 0x7fac1c151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fac21862640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac1c0c7eb0 0x7fac1c151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fac21862640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c14e070 con 0x7fac1c0ac7c0 2023-09-12T14:55:03.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fac21862640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c06c610 con 0x7fac1c06a640 2023-09-12T14:55:03.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fac21862640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c06c8f0 con 0x7fac1c0c7eb0 2023-09-12T14:55:03.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fac1a7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac1c0ac7c0 0x7fac1c151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fac1a7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac1c0ac7c0 0x7fac1c151520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38942/0 (socket says 172.21.15.131:38942) 2023-09-12T14:55:03.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fac1a7fc640 1 -- 172.21.15.131:0/2609442173 learned_addr learned my addr 172.21.15.131:0/2609442173 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:03.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f71b259c640 1 --2- 172.21.15.131:0/3138093374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71b40c9390 0x7f71b40c6bd0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f71a4000c10 tx=0x7f71a40a5750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fe9a77fe640 1 -- 172.21.15.131:0/1047406089 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fe9b80053e0 con 0x7fe9c00ccc50 2023-09-12T14:55:03.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fef0dbe0640 1 -- 172.21.15.131:0/2348644559 >> 172.21.15.131:0/2348644559 conn(0x7fef080d0a10 msgr2=0x7fef080d0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:03.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f6465a33640 1 -- 172.21.15.131:0/3830518603 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6444000fd0 con 0x7f6460156e80 2023-09-12T14:55:03.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f71a2ffd640 1 -- 172.21.15.131:0/3138093374 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4003800 con 0x7f71b40c9390 2023-09-12T14:55:03.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f71a2ffd640 1 -- 172.21.15.131:0/3138093374 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f71a4003960 con 0x7f71b40c9390 2023-09-12T14:55:03.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fe9a77fe640 1 --2- 172.21.15.131:0/1047406089 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe9ac065b50 0x7fe9ac068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fef0dbe0640 1 -- 172.21.15.131:0/2348644559 shutdown_connections 2023-09-12T14:55:03.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fef0dbe0640 1 -- 172.21.15.131:0/2348644559 wait complete. 2023-09-12T14:55:03.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f6c7c98c640 1 --2- 172.21.15.131:0/4182048866 >> [v2:172.21.15.131:6816/2354456273,v1:172.21.15.131:6817/2354456273] conn(0x7f6c5c0015f0 0x7f6c5c003ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f88f304c640 1 -- 172.21.15.131:0/125698334 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88ec0708b0 msgr2=0x7f88ec156fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f88f304c640 1 --2- 172.21.15.131:0/125698334 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88ec0708b0 0x7f88ec156fb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f88dc013f30 tx=0x7f88dc024d70 comp rx=0 tx=0).stop 2023-09-12T14:55:03.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f71b359e640 1 -- 172.21.15.131:0/3138093374 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b40c4f20 con 0x7f71b40c9390 2023-09-12T14:55:03.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f71a2ffd640 1 -- 172.21.15.131:0/3138093374 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4003d30 con 0x7f71b40c9390 2023-09-12T14:55:03.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f71b359e640 1 -- 172.21.15.131:0/3138093374 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b413a820 con 0x7f71b40c9390 2023-09-12T14:55:03.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fe9a77fe640 1 -- 172.21.15.131:0/1047406089 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9b8014070 con 0x7fe9c00ccc50 2023-09-12T14:55:03.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fe9a77fe640 1 --2- 172.21.15.131:0/1047406089 >> [v2:172.21.15.187:6801/221066879,v1:172.21.15.187:6803/221066879] conn(0x7fe9ac06dcf0 0x7fe9ac070110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fe9a77fe640 1 -- 172.21.15.131:0/1047406089 --> [v2:172.21.15.187:6801/221066879,v1:172.21.15.187:6803/221066879] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe9ac0707c0 con 0x7fe9ac06dcf0 2023-09-12T14:55:03.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fe9a77fe640 1 -- 172.21.15.131:0/1047406089 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9b8084800 con 0x7fe9c00ccc50 2023-09-12T14:55:03.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f6c7c98c640 1 -- 172.21.15.131:0/4182048866 --> [v2:172.21.15.131:6816/2354456273,v1:172.21.15.131:6817/2354456273] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6c5c006b50 con 0x7f6c5c0015f0 2023-09-12T14:55:03.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f88f304c640 1 -- 172.21.15.131:0/125698334 shutdown_connections 2023-09-12T14:55:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f88f304c640 1 --2- 172.21.15.131:0/125698334 >> v2:172.21.15.131:3300/0 conn(0x7f88ec06fe90 0x7f88ec0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f88f304c640 1 --2- 172.21.15.131:0/125698334 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88ec0708b0 0x7f88ec156fb0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f88f304c640 1 -- 172.21.15.131:0/125698334 >> 172.21.15.131:0/125698334 conn(0x7f88ec069240 msgr2=0x7f88ec069650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f88f304c640 1 -- 172.21.15.131:0/125698334 shutdown_connections 2023-09-12T14:55:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f88f304c640 1 -- 172.21.15.131:0/125698334 wait complete. 2023-09-12T14:55:03.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f6c7759e640 1 --2- 172.21.15.131:0/4182048866 >> [v2:172.21.15.131:6816/2354456273,v1:172.21.15.131:6817/2354456273] conn(0x7f6c5c0015f0 0x7f6c5c003ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fac1b7fe640 1 --2- 172.21.15.131:0/2609442173 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac1c0c7eb0 0x7fac1c151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fac1affd640 1 --2- 172.21.15.131:0/2609442173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac1c06a640 0x7fac1c0c3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f6c7759e640 1 --2- 172.21.15.131:0/4182048866 >> [v2:172.21.15.131:6816/2354456273,v1:172.21.15.131:6817/2354456273] conn(0x7f6c5c0015f0 0x7f6c5c003ab0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fe9beffd640 1 --2- 172.21.15.131:0/1047406089 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe9ac065b50 0x7fe9ac068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fef0dbe0640 1 Processor -- start 2023-09-12T14:55:03.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fe9beffd640 1 --2- 172.21.15.131:0/1047406089 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe9ac065b50 0x7fe9ac068010 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40b0000 tx=0x7fe9b401c350 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f71b359e640 1 -- 172.21.15.131:0/3138093374 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7198000fd0 con 0x7f71b40c9390 2023-09-12T14:55:03.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fef0dbe0640 1 -- start start 2023-09-12T14:55:03.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fef0dbe0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef080afc20 0x7fef0815c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fef0dbe0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef080ccbc0 0x7fef08154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fef0dbe0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef080c7ae0 0x7fef08155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fef0dbe0640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef08155390 con 0x7fef080ccbc0 2023-09-12T14:55:03.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fef0dbe0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef080c82c0 con 0x7fef080afc20 2023-09-12T14:55:03.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fef0dbe0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef080c85a0 con 0x7fef080c7ae0 2023-09-12T14:55:03.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fef07fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef080ccbc0 0x7fef08154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fef07fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef080ccbc0 0x7fef08154e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38966/0 (socket says 172.21.15.131:38966) 2023-09-12T14:55:03.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fef07fff640 1 -- 172.21.15.131:0/2558028399 learned_addr learned my addr 172.21.15.131:0/2558028399 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:03.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fef0cbde640 1 --2- 172.21.15.131:0/2558028399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef080afc20 0x7fef0815c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fef07fff640 1 -- 172.21.15.131:0/2558028399 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef080c7ae0 msgr2=0x7fef08155560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-12T14:55:03.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fef07fff640 1 --2- 172.21.15.131:0/2558028399 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef080c7ae0 0x7fef08155560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fef07fff640 1 -- 172.21.15.131:0/2558028399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef080afc20 msgr2=0x7fef0815c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fef07fff640 1 --2- 172.21.15.131:0/2558028399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef080afc20 0x7fef0815c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fef07fff640 1 -- 172.21.15.131:0/2558028399 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef000098e0 con 0x7fef080ccbc0 2023-09-12T14:55:03.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fef0cbde640 1 --2- 172.21.15.131:0/2558028399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef080afc20 0x7fef0815c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:55:03.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fef07fff640 1 --2- 172.21.15.131:0/2558028399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef080ccbc0 0x7fef08154e50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7feef400e7c0 tx=0x7feef400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f645cff9640 1 -- 172.21.15.131:0/3830518603 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f64580127e0 con 0x7f6460156e80 2023-09-12T14:55:03.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f645cff9640 1 --2- 172.21.15.131:0/3830518603 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6440065b20 0x7f6440067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f645cff9640 1 -- 172.21.15.131:0/3830518603 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6458084030 con 0x7f6460156e80 2023-09-12T14:55:03.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f645cff9640 1 --2- 172.21.15.131:0/3830518603 >> [v2:172.21.15.131:6808/3882516365,v1:172.21.15.131:6809/3882516365] conn(0x7f644006dcc0 0x7f64400700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f645cff9640 1 -- 172.21.15.131:0/3830518603 --> [v2:172.21.15.131:6808/3882516365,v1:172.21.15.131:6809/3882516365] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6440070790 con 0x7f644006dcc0 2023-09-12T14:55:03.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f645cff9640 1 -- 172.21.15.131:0/3830518603 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6458084480 con 0x7f6460156e80 2023-09-12T14:55:03.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f645effd640 1 --2- 172.21.15.131:0/3830518603 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6440065b20 0x7f6440067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f645effd640 1 --2- 172.21.15.131:0/3830518603 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6440065b20 0x7f6440067fe0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f64500af000 tx=0x7f645001b040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f64567fc640 1 --2- 172.21.15.131:0/3830518603 >> [v2:172.21.15.131:6808/3882516365,v1:172.21.15.131:6809/3882516365] conn(0x7f644006dcc0 0x7f64400700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fac1a7fc640 1 -- 172.21.15.131:0/2609442173 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac1c0c7eb0 msgr2=0x7fac1c151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fac1a7fc640 1 --2- 172.21.15.131:0/2609442173 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac1c0c7eb0 0x7fac1c151a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fac1a7fc640 1 -- 172.21.15.131:0/2609442173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac1c06a640 msgr2=0x7fac1c0c3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fac1a7fc640 1 --2- 172.21.15.131:0/2609442173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac1c06a640 0x7fac1c0c3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fac1a7fc640 1 -- 172.21.15.131:0/2609442173 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac0c009560 con 0x7fac1c0ac7c0 2023-09-12T14:55:03.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fac1b7fe640 1 --2- 172.21.15.131:0/2609442173 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac1c0c7eb0 0x7fac1c151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T14:55:03.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f6c6f7fe640 1 -- 172.21.15.131:0/4182048866 <== osd.3 v2:172.21.15.131:6816/2354456273 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f6c5c006b50 con 0x7f6c5c0015f0 2023-09-12T14:55:03.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f88f304c640 1 Processor -- start 2023-09-12T14:55:03.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7f88f304c640 1 -- start start 2023-09-12T14:55:03.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f71a2ffd640 1 -- 172.21.15.131:0/3138093374 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f71a40b4070 con 0x7f71b40c9390 2023-09-12T14:55:03.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f71a2ffd640 1 --2- 172.21.15.131:0/3138093374 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7194065b20 0x7f7194067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f71a2ffd640 1 -- 172.21.15.131:0/3138093374 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71a411e0b0 con 0x7f71b40c9390 2023-09-12T14:55:03.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f71b1d9b640 1 --2- 172.21.15.131:0/3138093374 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7194065b20 0x7f7194067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f71a2ffd640 1 --2- 172.21.15.131:0/3138093374 >> [v2:172.21.15.187:6824/2170805852,v1:172.21.15.187:6825/2170805852] conn(0x7f719406dcc0 0x7f71940700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f71a2ffd640 1 -- 172.21.15.131:0/3138093374 --> [v2:172.21.15.187:6824/2170805852,v1:172.21.15.187:6825/2170805852] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7194070790 con 0x7f719406dcc0 2023-09-12T14:55:03.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f71a2ffd640 1 -- 172.21.15.131:0/3138093374 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71a411e500 con 0x7f71b40c9390 2023-09-12T14:55:03.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f71b2d9d640 1 --2- 172.21.15.131:0/3138093374 >> [v2:172.21.15.187:6824/2170805852,v1:172.21.15.187:6825/2170805852] conn(0x7f719406dcc0 0x7f71940700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f71b1d9b640 1 --2- 172.21.15.131:0/3138093374 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7194065b20 0x7f7194067fe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f71a8005fd0 tx=0x7f71a8005e60 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.821+0000 7fe9be7fc640 1 --2- 172.21.15.131:0/1047406089 >> [v2:172.21.15.187:6801/221066879,v1:172.21.15.187:6803/221066879] conn(0x7fe9ac06dcf0 0x7fe9ac070110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fe9be7fc640 1 --2- 172.21.15.131:0/1047406089 >> [v2:172.21.15.187:6801/221066879,v1:172.21.15.187:6803/221066879] conn(0x7fe9ac06dcf0 0x7fe9ac070110 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fe804ff9640 1 -- 172.21.15.131:0/1176752724 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fe7f801c460 con 0x7fe80806cf70 2023-09-12T14:55:03.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fe804ff9640 1 --2- 172.21.15.131:0/1176752724 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe7f4065b70 0x7fe7f4068030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fe804ff9640 1 -- 172.21.15.131:0/1176752724 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7f8083ee0 con 0x7fe80806cf70 2023-09-12T14:55:03.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fe804ff9640 1 --2- 172.21.15.131:0/1176752724 >> [v2:172.21.15.187:6816/827490193,v1:172.21.15.187:6817/827490193] conn(0x7fe7f406dd90 0x7fe7f40701b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fe804ff9640 1 -- 172.21.15.131:0/1176752724 --> [v2:172.21.15.187:6816/827490193,v1:172.21.15.187:6817/827490193] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe7f4070860 con 0x7fe7f406dd90 2023-09-12T14:55:03.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fe804ff9640 1 -- 172.21.15.131:0/1176752724 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7f8084290 con 0x7fe80806cf70 2023-09-12T14:55:03.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fe8077fe640 1 --2- 172.21.15.131:0/1176752724 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe7f4065b70 0x7fe7f4068030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fe80ca18640 1 --2- 172.21.15.131:0/1176752724 >> [v2:172.21.15.187:6816/827490193,v1:172.21.15.187:6817/827490193] conn(0x7fe7f406dd90 0x7fe7f40701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f71b2d9d640 1 --2- 172.21.15.131:0/3138093374 >> [v2:172.21.15.187:6824/2170805852,v1:172.21.15.187:6825/2170805852] conn(0x7f719406dcc0 0x7f71940700e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fef057fa640 1 -- 172.21.15.131:0/2558028399 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef40095b0 con 0x7fef080ccbc0 2023-09-12T14:55:03.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fef057fa640 1 -- 172.21.15.131:0/2558028399 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feef400eed0 con 0x7fef080ccbc0 2023-09-12T14:55:03.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fef0dbe0640 1 -- 172.21.15.131:0/2558028399 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef080c8880 con 0x7fef080ccbc0 2023-09-12T14:55:03.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fef057fa640 1 -- 172.21.15.131:0/2558028399 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef4005020 con 0x7fef080ccbc0 2023-09-12T14:55:03.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fef0dbe0640 1 -- 172.21.15.131:0/2558028399 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef080c8e40 con 0x7fef080ccbc0 2023-09-12T14:55:03.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fef0dbe0640 1 -- 172.21.15.131:0/2558028399 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fef0804e110 con 0x7fef080ccbc0 2023-09-12T14:55:03.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f64567fc640 1 --2- 172.21.15.131:0/3830518603 >> [v2:172.21.15.131:6808/3882516365,v1:172.21.15.131:6809/3882516365] conn(0x7f644006dcc0 0x7f64400700e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fe9a77fe640 1 -- 172.21.15.131:0/1047406089 <== osd.6 v2:172.21.15.187:6801/221066879 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fe9ac0707c0 con 0x7fe9ac06dcf0 2023-09-12T14:55:03.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fe80ca18640 1 --2- 172.21.15.131:0/1176752724 >> [v2:172.21.15.187:6816/827490193,v1:172.21.15.187:6817/827490193] conn(0x7fe7f406dd90 0x7fe7f40701b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fef057fa640 1 -- 172.21.15.131:0/2558028399 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7feef4005180 con 0x7fef080ccbc0 2023-09-12T14:55:03.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fe8077fe640 1 --2- 172.21.15.131:0/1176752724 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe7f4065b70 0x7fe7f4068030 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe8000af000 tx=0x7fe80001b310 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7fef057fa640 1 --2- 172.21.15.131:0/2558028399 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7feeec065b50 0x7feeec068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.829+0000 7fef057fa640 1 -- 172.21.15.131:0/2558028399 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feef4014070 con 0x7fef080ccbc0 2023-09-12T14:55:03.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.829+0000 7fef057fa640 1 --2- 172.21.15.131:0/2558028399 >> [v2:172.21.15.131:6800/2063242002,v1:172.21.15.131:6801/2063242002] conn(0x7feeec06dcf0 0x7feeec070110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.829+0000 7fef057fa640 1 -- 172.21.15.131:0/2558028399 --> [v2:172.21.15.131:6800/2063242002,v1:172.21.15.131:6801/2063242002] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feeec0707c0 con 0x7feeec06dcf0 2023-09-12T14:55:03.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.829+0000 7fef057fa640 1 -- 172.21.15.131:0/2558028399 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7feef4084580 con 0x7fef080ccbc0 2023-09-12T14:55:03.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.829+0000 7f645cff9640 1 -- 172.21.15.131:0/3830518603 <== osd.1 v2:172.21.15.131:6808/3882516365 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f6440070790 con 0x7f644006dcc0 2023-09-12T14:55:03.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.829+0000 7fac1a7fc640 1 --2- 172.21.15.131:0/2609442173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac1c0ac7c0 0x7fac1c151520 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fac1400caf0 tx=0x7fac14002130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f88f304c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88ec06fa70 0x7f88ec150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f88f304c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88ec06fe90 0x7f88ec0c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f88f204a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88ec06fa70 0x7f88ec150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f88f204a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88ec06fa70 0x7f88ec150980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:53814/0 (socket says 172.21.15.131:53814) 2023-09-12T14:55:03.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f88f204a640 1 -- 172.21.15.131:0/1300077331 learned_addr learned my addr 172.21.15.131:0/1300077331 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:03.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f88f304c640 1 --2- 172.21.15.131:0/1300077331 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88ec0708b0 0x7f88ec0cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f88f304c640 1 -- 172.21.15.131:0/1300077331 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec0c6aa0 con 0x7f88ec06fe90 2023-09-12T14:55:03.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f88f304c640 1 -- 172.21.15.131:0/1300077331 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec153d70 con 0x7f88ec06fa70 2023-09-12T14:55:03.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f88f304c640 1 -- 172.21.15.131:0/1300077331 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec154010 con 0x7f88ec0708b0 2023-09-12T14:55:03.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f88f1849640 1 --2- 172.21.15.131:0/1300077331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88ec06fe90 0x7f88ec0c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f88f284b640 1 --2- 172.21.15.131:0/1300077331 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88ec0708b0 0x7f88ec0cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f88f204a640 1 -- 172.21.15.131:0/1300077331 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88ec0708b0 msgr2=0x7f88ec0cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f88f204a640 1 --2- 172.21.15.131:0/1300077331 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88ec0708b0 0x7f88ec0cb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f88f204a640 1 -- 172.21.15.131:0/1300077331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88ec06fe90 msgr2=0x7f88ec0c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f88f204a640 1 --2- 172.21.15.131:0/1300077331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88ec06fe90 0x7f88ec0c6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f88f204a640 1 -- 172.21.15.131:0/1300077331 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88dc013b90 con 0x7f88ec06fa70 2023-09-12T14:55:03.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f88f1849640 1 --2- 172.21.15.131:0/1300077331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88ec06fe90 0x7f88ec0c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-12T14:55:03.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f88f204a640 1 --2- 172.21.15.131:0/1300077331 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88ec06fa70 0x7f88ec150980 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f88dc023420 tx=0x7f88dc005e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f88e2ffd640 1 -- 172.21.15.131:0/1300077331 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88dc003980 con 0x7f88ec06fa70 2023-09-12T14:55:03.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f88f304c640 1 -- 172.21.15.131:0/1300077331 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88ec1542f0 con 0x7f88ec06fa70 2023-09-12T14:55:03.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f88f304c640 1 -- 172.21.15.131:0/1300077331 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88ec0c3fe0 con 0x7f88ec06fa70 2023-09-12T14:55:03.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.825+0000 7f88f304c640 1 -- 172.21.15.131:0/1300077331 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f88ec04e110 con 0x7f88ec06fa70 2023-09-12T14:55:03.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.829+0000 7f88e2ffd640 1 -- 172.21.15.131:0/1300077331 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f88dc003ae0 con 0x7f88ec06fa70 2023-09-12T14:55:03.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.829+0000 7f88e2ffd640 1 -- 172.21.15.131:0/1300077331 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88dc003eb0 con 0x7f88ec06fa70 2023-09-12T14:55:03.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.829+0000 7f71a2ffd640 1 -- 172.21.15.131:0/3138093374 <== osd.5 v2:172.21.15.187:6824/2170805852 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f7194070790 con 0x7f719406dcc0 2023-09-12T14:55:03.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.829+0000 7fef0d3df640 1 --2- 172.21.15.131:0/2558028399 >> [v2:172.21.15.131:6800/2063242002,v1:172.21.15.131:6801/2063242002] conn(0x7feeec06dcf0 0x7feeec070110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.829+0000 7fef0cbde640 1 --2- 172.21.15.131:0/2558028399 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7feeec065b50 0x7feeec068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.829+0000 7fef0cbde640 1 --2- 172.21.15.131:0/2558028399 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7feeec065b50 0x7feeec068010 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fef080678a0 tx=0x7feefc009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.829+0000 7fef0d3df640 1 --2- 172.21.15.131:0/2558028399 >> [v2:172.21.15.131:6800/2063242002,v1:172.21.15.131:6801/2063242002] conn(0x7feeec06dcf0 0x7feeec070110 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.829+0000 7fef057fa640 1 -- 172.21.15.131:0/2558028399 <== osd.0 v2:172.21.15.131:6800/2063242002 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7feeec0707c0 con 0x7feeec06dcf0 2023-09-12T14:55:03.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.829+0000 7fac03fff640 1 -- 172.21.15.131:0/2609442173 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1400ab70 con 0x7fac1c0ac7c0 2023-09-12T14:55:03.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.829+0000 7fac03fff640 1 -- 172.21.15.131:0/2609442173 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fac1400adc0 con 0x7fac1c0ac7c0 2023-09-12T14:55:03.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.829+0000 7fac03fff640 1 -- 172.21.15.131:0/2609442173 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac14012680 con 0x7fac1c0ac7c0 2023-09-12T14:55:03.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.829+0000 7fac21862640 1 -- 172.21.15.131:0/2609442173 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac1c06cbd0 con 0x7fac1c0ac7c0 2023-09-12T14:55:03.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.829+0000 7fac21862640 1 -- 172.21.15.131:0/2609442173 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac1c0c3880 con 0x7fac1c0ac7c0 2023-09-12T14:55:03.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.829+0000 7f88e2ffd640 1 -- 172.21.15.131:0/1300077331 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f88dc0bc070 con 0x7f88ec06fa70 2023-09-12T14:55:03.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.833+0000 7fac03fff640 1 -- 172.21.15.131:0/2609442173 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fac1401b410 con 0x7fac1c0ac7c0 2023-09-12T14:55:03.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.833+0000 7fac03fff640 1 --2- 172.21.15.131:0/2609442173 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fac08065bf0 0x7fac080680b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.833+0000 7fac03fff640 1 -- 172.21.15.131:0/2609442173 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac140835a0 con 0x7fac1c0ac7c0 2023-09-12T14:55:03.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.833+0000 7fac1affd640 1 --2- 172.21.15.131:0/2609442173 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fac08065bf0 0x7fac080680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.829+0000 7f88e2ffd640 1 --2- 172.21.15.131:0/1300077331 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f88d0065b20 0x7f88d0067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.833+0000 7f88e2ffd640 1 -- 172.21.15.131:0/1300077331 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88dc11cf10 con 0x7f88ec06fa70 2023-09-12T14:55:03.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.833+0000 7f88e2ffd640 1 --2- 172.21.15.131:0/1300077331 >> [v2:172.21.15.187:6800/1649621344,v1:172.21.15.187:6802/1649621344] conn(0x7f88d006dcc0 0x7f88d00700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.833+0000 7f88e2ffd640 1 -- 172.21.15.131:0/1300077331 --> [v2:172.21.15.187:6800/1649621344,v1:172.21.15.187:6802/1649621344] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f88d0070790 con 0x7f88d006dcc0 2023-09-12T14:55:03.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.833+0000 7f88e2ffd640 1 -- 172.21.15.131:0/1300077331 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88dc11d280 con 0x7f88ec06fa70 2023-09-12T14:55:03.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.833+0000 7fe804ff9640 1 -- 172.21.15.131:0/1176752724 <== osd.7 v2:172.21.15.187:6816/827490193 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fe7f4070860 con 0x7fe7f406dd90 2023-09-12T14:55:03.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.833+0000 7fac1affd640 1 --2- 172.21.15.131:0/2609442173 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fac08065bf0 0x7fac080680b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fac0c0b2000 tx=0x7fac0c01d830 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.837+0000 7f88f1849640 1 --2- 172.21.15.131:0/1300077331 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f88d0065b20 0x7f88d0067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.841+0000 7f88f284b640 1 --2- 172.21.15.131:0/1300077331 >> [v2:172.21.15.187:6800/1649621344,v1:172.21.15.187:6802/1649621344] conn(0x7f88d006dcc0 0x7f88d00700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.841+0000 7f88f1849640 1 --2- 172.21.15.131:0/1300077331 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f88d0065b20 0x7f88d0067fe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f88ec15b560 tx=0x7f88e4009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.841+0000 7fac21862640 1 --2- 172.21.15.131:0/2609442173 >> [v2:172.21.15.131:6824/3891085905,v1:172.21.15.131:6825/3891085905] conn(0x7fabfc0015f0 0x7fabfc003ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:03.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.841+0000 7fac21862640 1 -- 172.21.15.131:0/2609442173 --> [v2:172.21.15.131:6824/3891085905,v1:172.21.15.131:6825/3891085905] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fabfc006b50 con 0x7fabfc0015f0 2023-09-12T14:55:03.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.841+0000 7f88f284b640 1 --2- 172.21.15.131:0/1300077331 >> [v2:172.21.15.187:6800/1649621344,v1:172.21.15.187:6802/1649621344] conn(0x7f88d006dcc0 0x7f88d00700e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.841+0000 7fac1b7fe640 1 --2- 172.21.15.131:0/2609442173 >> [v2:172.21.15.131:6824/3891085905,v1:172.21.15.131:6825/3891085905] conn(0x7fabfc0015f0 0x7fabfc003ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:03.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.841+0000 7fac1b7fe640 1 --2- 172.21.15.131:0/2609442173 >> [v2:172.21.15.131:6824/3891085905,v1:172.21.15.131:6825/3891085905] conn(0x7fabfc0015f0 0x7fabfc003ab0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:03.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.841+0000 7fac03fff640 1 -- 172.21.15.131:0/2609442173 <== osd.2 v2:172.21.15.131:6824/3891085905 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fabfc006b50 con 0x7fabfc0015f0 2023-09-12T14:55:03.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.841+0000 7f88e2ffd640 1 -- 172.21.15.131:0/1300077331 <== osd.4 v2:172.21.15.187:6800/1649621344 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f88d0070790 con 0x7f88d006dcc0 2023-09-12T14:55:03.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.845+0000 7fe9c5590640 1 -- 172.21.15.131:0/1047406089 --> [v2:172.21.15.187:6801/221066879,v1:172.21.15.187:6803/221066879] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe9c0159980 con 0x7fe9ac06dcf0 2023-09-12T14:55:03.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.845+0000 7fe9a77fe640 1 -- 172.21.15.131:0/1047406089 <== osd.6 v2:172.21.15.187:6801/221066879 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe9c0159980 con 0x7fe9ac06dcf0 2023-09-12T14:55:03.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.845+0000 7fef0dbe0640 1 -- 172.21.15.131:0/2558028399 --> [v2:172.21.15.131:6800/2063242002,v1:172.21.15.131:6801/2063242002] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fef080c9060 con 0x7feeec06dcf0 2023-09-12T14:55:03.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.849+0000 7fef057fa640 1 -- 172.21.15.131:0/2558028399 <== osd.0 v2:172.21.15.131:6800/2063242002 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fef080c9060 con 0x7feeec06dcf0 2023-09-12T14:55:03.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.849+0000 7fef0dbe0640 1 -- 172.21.15.131:0/2558028399 >> [v2:172.21.15.131:6800/2063242002,v1:172.21.15.131:6801/2063242002] conn(0x7feeec06dcf0 msgr2=0x7feeec070110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.849+0000 7fef0dbe0640 1 --2- 172.21.15.131:0/2558028399 >> [v2:172.21.15.131:6800/2063242002,v1:172.21.15.131:6801/2063242002] conn(0x7feeec06dcf0 0x7feeec070110 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fe80e47b640 1 -- 172.21.15.131:0/1176752724 --> [v2:172.21.15.187:6816/827490193,v1:172.21.15.187:6817/827490193] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe8080c2480 con 0x7fe7f406dd90 2023-09-12T14:55:03.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fe804ff9640 1 -- 172.21.15.131:0/1176752724 <== osd.7 v2:172.21.15.187:6816/827490193 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe8080c2480 con 0x7fe7f406dd90 2023-09-12T14:55:03.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fef0dbe0640 1 -- 172.21.15.131:0/2558028399 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7feeec065b50 msgr2=0x7feeec068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fef0dbe0640 1 --2- 172.21.15.131:0/2558028399 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7feeec065b50 0x7feeec068010 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fef080678a0 tx=0x7feefc009040 comp rx=0 tx=0).stop 2023-09-12T14:55:03.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fef0dbe0640 1 -- 172.21.15.131:0/2558028399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef080ccbc0 msgr2=0x7fef08154e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fef0dbe0640 1 --2- 172.21.15.131:0/2558028399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef080ccbc0 0x7fef08154e50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7feef400e7c0 tx=0x7feef400ec80 comp rx=0 tx=0).stop 2023-09-12T14:55:03.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fef0d3df640 1 -- 172.21.15.131:0/2558028399 reap_dead start 2023-09-12T14:55:03.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fef0dbe0640 1 -- 172.21.15.131:0/2558028399 shutdown_connections 2023-09-12T14:55:03.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fef0dbe0640 1 -- 172.21.15.131:0/2558028399 >> 172.21.15.131:0/2558028399 conn(0x7fef080d0a10 msgr2=0x7fef080b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:03.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fe9c5590640 1 -- 172.21.15.131:0/1047406089 >> [v2:172.21.15.187:6801/221066879,v1:172.21.15.187:6803/221066879] conn(0x7fe9ac06dcf0 msgr2=0x7fe9ac070110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fe9c5590640 1 --2- 172.21.15.131:0/1047406089 >> [v2:172.21.15.187:6801/221066879,v1:172.21.15.187:6803/221066879] conn(0x7fe9ac06dcf0 0x7fe9ac070110 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fe9c5590640 1 -- 172.21.15.131:0/1047406089 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe9ac065b50 msgr2=0x7fe9ac068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fe9c5590640 1 --2- 172.21.15.131:0/1047406089 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe9ac065b50 0x7fe9ac068010 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40b0000 tx=0x7fe9b401c350 comp rx=0 tx=0).stop 2023-09-12T14:55:03.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fe9c5590640 1 -- 172.21.15.131:0/1047406089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9c00ccc50 msgr2=0x7fe9c0145450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fe9c5590640 1 --2- 172.21.15.131:0/1047406089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9c00ccc50 0x7fe9c0145450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800eaf0 tx=0x7fe9b800c2d0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fef0dbe0640 1 -- 172.21.15.131:0/2558028399 shutdown_connections 2023-09-12T14:55:03.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fef0dbe0640 1 -- 172.21.15.131:0/2558028399 wait complete. 2023-09-12T14:55:03.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7f6c7c98c640 1 -- 172.21.15.131:0/4182048866 --> [v2:172.21.15.131:6816/2354456273,v1:172.21.15.131:6817/2354456273] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6c5c005c30 con 0x7f6c5c0015f0 2023-09-12T14:55:03.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fe9bf7fe640 1 -- 172.21.15.131:0/1047406089 reap_dead start 2023-09-12T14:55:03.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fe9c5590640 1 -- 172.21.15.131:0/1047406089 shutdown_connections 2023-09-12T14:55:03.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fe9c5590640 1 -- 172.21.15.131:0/1047406089 >> 172.21.15.131:0/1047406089 conn(0x7fe9c006b3c0 msgr2=0x7fe9c014c150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:03.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fe9c5590640 1 -- 172.21.15.131:0/1047406089 shutdown_connections 2023-09-12T14:55:03.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fe9c5590640 1 -- 172.21.15.131:0/1047406089 wait complete. 2023-09-12T14:55:03.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fe80e47b640 1 -- 172.21.15.131:0/1176752724 >> [v2:172.21.15.187:6816/827490193,v1:172.21.15.187:6817/827490193] conn(0x7fe7f406dd90 msgr2=0x7fe7f40701b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fe80e47b640 1 --2- 172.21.15.131:0/1176752724 >> [v2:172.21.15.187:6816/827490193,v1:172.21.15.187:6817/827490193] conn(0x7fe7f406dd90 0x7fe7f40701b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7f6c6f7fe640 1 -- 172.21.15.131:0/4182048866 <== osd.3 v2:172.21.15.131:6816/2354456273 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6c5c005c30 con 0x7f6c5c0015f0 2023-09-12T14:55:03.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fe80e47b640 1 -- 172.21.15.131:0/1176752724 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe7f4065b70 msgr2=0x7fe7f4068030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fe80e47b640 1 --2- 172.21.15.131:0/1176752724 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe7f4065b70 0x7fe7f4068030 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe8000af000 tx=0x7fe80001b310 comp rx=0 tx=0).stop 2023-09-12T14:55:03.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fe80e47b640 1 -- 172.21.15.131:0/1176752724 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe80806cf70 msgr2=0x7fe80815d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.853+0000 7fe80e47b640 1 --2- 172.21.15.131:0/1176752724 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe80806cf70 0x7fe80815d680 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe7f800c410 tx=0x7fe7f800c8d0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7fe80ca18640 1 -- 172.21.15.131:0/1176752724 reap_dead start 2023-09-12T14:55:03.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f6c6d7fa640 1 -- 172.21.15.131:0/4182048866 >> [v2:172.21.15.131:6816/2354456273,v1:172.21.15.131:6817/2354456273] conn(0x7f6c5c0015f0 msgr2=0x7f6c5c003ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f6c6d7fa640 1 --2- 172.21.15.131:0/4182048866 >> [v2:172.21.15.131:6816/2354456273,v1:172.21.15.131:6817/2354456273] conn(0x7f6c5c0015f0 0x7f6c5c003ab0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7fe80e47b640 1 -- 172.21.15.131:0/1176752724 shutdown_connections 2023-09-12T14:55:03.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7fe80e47b640 1 -- 172.21.15.131:0/1176752724 >> 172.21.15.131:0/1176752724 conn(0x7fe80806b470 msgr2=0x7fe8080ac130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:03.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f6c6d7fa640 1 -- 172.21.15.131:0/4182048866 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6c58065b20 msgr2=0x7f6c58067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f6c6d7fa640 1 --2- 172.21.15.131:0/4182048866 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6c58065b20 0x7f6c58067fe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6c60003cf0 tx=0x7f6c600079c0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f6c6d7fa640 1 -- 172.21.15.131:0/4182048866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c780cb350 msgr2=0x7f6c7806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f6c6d7fa640 1 --2- 172.21.15.131:0/4182048866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c780cb350 0x7f6c7806d3e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6c680ac000 tx=0x7f6c680a57d0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f6c7759e640 1 -- 172.21.15.131:0/4182048866 reap_dead start 2023-09-12T14:55:03.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f6c6d7fa640 1 -- 172.21.15.131:0/4182048866 shutdown_connections 2023-09-12T14:55:03.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f6c6d7fa640 1 -- 172.21.15.131:0/4182048866 >> 172.21.15.131:0/4182048866 conn(0x7f6c780ce870 msgr2=0x7f6c780cbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:03.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7fe80e47b640 1 -- 172.21.15.131:0/1176752724 shutdown_connections 2023-09-12T14:55:03.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7fe80e47b640 1 -- 172.21.15.131:0/1176752724 wait complete. 2023-09-12T14:55:03.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f71b359e640 1 -- 172.21.15.131:0/3138093374 --> [v2:172.21.15.187:6824/2170805852,v1:172.21.15.187:6825/2170805852] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7198002cc0 con 0x7f719406dcc0 2023-09-12T14:55:03.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f6465a33640 1 -- 172.21.15.131:0/3830518603 --> [v2:172.21.15.131:6808/3882516365,v1:172.21.15.131:6809/3882516365] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6444002cc0 con 0x7f644006dcc0 2023-09-12T14:55:03.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f645cff9640 1 -- 172.21.15.131:0/3830518603 <== osd.1 v2:172.21.15.131:6808/3882516365 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6444002cc0 con 0x7f644006dcc0 2023-09-12T14:55:03.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f71a2ffd640 1 -- 172.21.15.131:0/3138093374 <== osd.5 v2:172.21.15.187:6824/2170805852 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7198002cc0 con 0x7f719406dcc0 2023-09-12T14:55:03.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f71b359e640 1 -- 172.21.15.131:0/3138093374 >> [v2:172.21.15.187:6824/2170805852,v1:172.21.15.187:6825/2170805852] conn(0x7f719406dcc0 msgr2=0x7f71940700e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f71b359e640 1 --2- 172.21.15.131:0/3138093374 >> [v2:172.21.15.187:6824/2170805852,v1:172.21.15.187:6825/2170805852] conn(0x7f719406dcc0 0x7f71940700e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f71b359e640 1 -- 172.21.15.131:0/3138093374 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7194065b20 msgr2=0x7f7194067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f71b359e640 1 --2- 172.21.15.131:0/3138093374 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7194065b20 0x7f7194067fe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f71a8005fd0 tx=0x7f71a8005e60 comp rx=0 tx=0).stop 2023-09-12T14:55:03.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f6465a33640 1 -- 172.21.15.131:0/3830518603 >> [v2:172.21.15.131:6808/3882516365,v1:172.21.15.131:6809/3882516365] conn(0x7f644006dcc0 msgr2=0x7f64400700e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f6465a33640 1 --2- 172.21.15.131:0/3830518603 >> [v2:172.21.15.131:6808/3882516365,v1:172.21.15.131:6809/3882516365] conn(0x7f644006dcc0 0x7f64400700e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f71b359e640 1 -- 172.21.15.131:0/3138093374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71b40c9390 msgr2=0x7f71b40c6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f71b359e640 1 --2- 172.21.15.131:0/3138093374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71b40c9390 0x7f71b40c6bd0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f71a4000c10 tx=0x7f71a40a5750 comp rx=0 tx=0).stop 2023-09-12T14:55:03.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f71b2d9d640 1 -- 172.21.15.131:0/3138093374 reap_dead start 2023-09-12T14:55:03.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f71b359e640 1 -- 172.21.15.131:0/3138093374 shutdown_connections 2023-09-12T14:55:03.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f71b359e640 1 -- 172.21.15.131:0/3138093374 >> 172.21.15.131:0/3138093374 conn(0x7f71b4069240 msgr2=0x7f71b4072320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:03.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f6465a33640 1 -- 172.21.15.131:0/3830518603 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6440065b20 msgr2=0x7f6440067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f6465a33640 1 --2- 172.21.15.131:0/3830518603 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6440065b20 0x7f6440067fe0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f64500af000 tx=0x7f645001b040 comp rx=0 tx=0).stop 2023-09-12T14:55:03.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f6465a33640 1 -- 172.21.15.131:0/3830518603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6460156e80 msgr2=0x7f6460167ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f6465a33640 1 --2- 172.21.15.131:0/3830518603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6460156e80 0x7f6460167ce0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f645800cb70 tx=0x7f645800cc60 comp rx=0 tx=0).stop 2023-09-12T14:55:03.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f645f7fe640 1 -- 172.21.15.131:0/3830518603 reap_dead start 2023-09-12T14:55:03.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.861+0000 7f6465a33640 1 -- 172.21.15.131:0/3830518603 shutdown_connections 2023-09-12T14:55:03.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.861+0000 7f6465a33640 1 -- 172.21.15.131:0/3830518603 >> 172.21.15.131:0/3830518603 conn(0x7f6460069840 msgr2=0x7f64600ab9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:03.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f71b359e640 1 -- 172.21.15.131:0/3138093374 shutdown_connections 2023-09-12T14:55:03.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.857+0000 7f71b359e640 1 -- 172.21.15.131:0/3138093374 wait complete. 2023-09-12T14:55:03.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.861+0000 7f6465a33640 1 -- 172.21.15.131:0/3830518603 shutdown_connections 2023-09-12T14:55:03.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.861+0000 7f6465a33640 1 -- 172.21.15.131:0/3830518603 wait complete. 2023-09-12T14:55:03.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.861+0000 7f88f304c640 1 -- 172.21.15.131:0/1300077331 --> [v2:172.21.15.187:6800/1649621344,v1:172.21.15.187:6802/1649621344] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f88ec066540 con 0x7f88d006dcc0 2023-09-12T14:55:03.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.861+0000 7f88e2ffd640 1 -- 172.21.15.131:0/1300077331 <== osd.4 v2:172.21.15.187:6800/1649621344 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f88ec066540 con 0x7f88d006dcc0 2023-09-12T14:55:03.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.861+0000 7f88f304c640 1 -- 172.21.15.131:0/1300077331 >> [v2:172.21.15.187:6800/1649621344,v1:172.21.15.187:6802/1649621344] conn(0x7f88d006dcc0 msgr2=0x7f88d00700e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.861+0000 7f88f304c640 1 --2- 172.21.15.131:0/1300077331 >> [v2:172.21.15.187:6800/1649621344,v1:172.21.15.187:6802/1649621344] conn(0x7f88d006dcc0 0x7f88d00700e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.861+0000 7f88f304c640 1 -- 172.21.15.131:0/1300077331 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f88d0065b20 msgr2=0x7f88d0067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.861+0000 7f88f304c640 1 --2- 172.21.15.131:0/1300077331 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f88d0065b20 0x7f88d0067fe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f88ec15b560 tx=0x7f88e4009040 comp rx=0 tx=0).stop 2023-09-12T14:55:03.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.861+0000 7f88f304c640 1 -- 172.21.15.131:0/1300077331 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88ec06fa70 msgr2=0x7f88ec150980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.861+0000 7f88f304c640 1 --2- 172.21.15.131:0/1300077331 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88ec06fa70 0x7f88ec150980 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f88dc023420 tx=0x7f88dc005e70 comp rx=0 tx=0).stop 2023-09-12T14:55:03.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.861+0000 7f88f284b640 1 -- 172.21.15.131:0/1300077331 reap_dead start 2023-09-12T14:55:03.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.861+0000 7f88f304c640 1 -- 172.21.15.131:0/1300077331 shutdown_connections 2023-09-12T14:55:03.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.861+0000 7f88f304c640 1 -- 172.21.15.131:0/1300077331 >> 172.21.15.131:0/1300077331 conn(0x7f88ec069240 msgr2=0x7f88ec069970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:03.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.861+0000 7f88f304c640 1 -- 172.21.15.131:0/1300077331 shutdown_connections 2023-09-12T14:55:03.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.861+0000 7f88f304c640 1 -- 172.21.15.131:0/1300077331 wait complete. 2023-09-12T14:55:03.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.865+0000 7f6c6d7fa640 1 -- 172.21.15.131:0/4182048866 shutdown_connections 2023-09-12T14:55:03.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.865+0000 7f6c6d7fa640 1 -- 172.21.15.131:0/4182048866 wait complete. 2023-09-12T14:55:03.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.877+0000 7fac21862640 1 -- 172.21.15.131:0/2609442173 --> [v2:172.21.15.131:6824/3891085905,v1:172.21.15.131:6825/3891085905] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fabfc005c30 con 0x7fabfc0015f0 2023-09-12T14:55:03.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.877+0000 7fac03fff640 1 -- 172.21.15.131:0/2609442173 <== osd.2 v2:172.21.15.131:6824/3891085905 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fabfc005c30 con 0x7fabfc0015f0 2023-09-12T14:55:03.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.885+0000 7fac01ffb640 1 -- 172.21.15.131:0/2609442173 >> [v2:172.21.15.131:6824/3891085905,v1:172.21.15.131:6825/3891085905] conn(0x7fabfc0015f0 msgr2=0x7fabfc003ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.885+0000 7fac01ffb640 1 --2- 172.21.15.131:0/2609442173 >> [v2:172.21.15.131:6824/3891085905,v1:172.21.15.131:6825/3891085905] conn(0x7fabfc0015f0 0x7fabfc003ab0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:03.936 INFO:teuthology.orchestra.run.smithi131.stdout:55834574854 2023-09-12T14:55:03.936 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-09-12T14:55:03.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.885+0000 7fac01ffb640 1 -- 172.21.15.131:0/2609442173 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fac08065bf0 msgr2=0x7fac080680b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.885+0000 7fac01ffb640 1 --2- 172.21.15.131:0/2609442173 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fac08065bf0 0x7fac080680b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fac0c0b2000 tx=0x7fac0c01d830 comp rx=0 tx=0).stop 2023-09-12T14:55:03.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.885+0000 7fac01ffb640 1 -- 172.21.15.131:0/2609442173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac1c0ac7c0 msgr2=0x7fac1c151520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:03.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.885+0000 7fac01ffb640 1 --2- 172.21.15.131:0/2609442173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac1c0ac7c0 0x7fac1c151520 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fac1400caf0 tx=0x7fac14002130 comp rx=0 tx=0).stop 2023-09-12T14:55:03.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.885+0000 7fac1b7fe640 1 -- 172.21.15.131:0/2609442173 reap_dead start 2023-09-12T14:55:03.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.885+0000 7fac01ffb640 1 -- 172.21.15.131:0/2609442173 shutdown_connections 2023-09-12T14:55:03.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.885+0000 7fac01ffb640 1 -- 172.21.15.131:0/2609442173 >> 172.21.15.131:0/2609442173 conn(0x7fac1c0cd150 msgr2=0x7fac1c070a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:03.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.885+0000 7fac01ffb640 1 -- 172.21.15.131:0/2609442173 shutdown_connections 2023-09-12T14:55:03.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:03.885+0000 7fac01ffb640 1 -- 172.21.15.131:0/2609442173 wait complete. 2023-09-12T14:55:03.940 INFO:teuthology.orchestra.run.smithi131.stdout:55834574854 2023-09-12T14:55:03.940 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-12T14:55:03.941 INFO:teuthology.orchestra.run.smithi131.stdout:55834574854 2023-09-12T14:55:03.941 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-09-12T14:55:03.942 INFO:teuthology.orchestra.run.smithi131.stdout:55834574854 2023-09-12T14:55:03.943 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-09-12T14:55:03.943 INFO:teuthology.orchestra.run.smithi131.stdout:55834574856 2023-09-12T14:55:03.944 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-09-12T14:55:03.945 INFO:teuthology.orchestra.run.smithi131.stdout:55834574854 2023-09-12T14:55:03.945 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-12T14:55:03.946 INFO:teuthology.orchestra.run.smithi131.stdout:55834574854 2023-09-12T14:55:03.946 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-12T14:55:03.947 INFO:teuthology.orchestra.run.smithi131.stdout:55834574853 2023-09-12T14:55:03.948 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-12T14:55:04.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.149+0000 7fe44bce8640 1 -- 172.21.15.131:0/340081121 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4440ccde0 msgr2=0x7fe4440ac150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.149+0000 7fe44bce8640 1 --2- 172.21.15.131:0/340081121 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4440ccde0 0x7fe4440ac150 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe434009840 tx=0x7fe43401c7d0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.149+0000 7fe44bce8640 1 -- 172.21.15.131:0/340081121 shutdown_connections 2023-09-12T14:55:04.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.149+0000 7fe44bce8640 1 --2- 172.21.15.131:0/340081121 >> v2:172.21.15.187:3300/0 conn(0x7fe44406ee20 0x7fe444154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.149+0000 7fe44bce8640 1 --2- 172.21.15.131:0/340081121 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4440ccde0 0x7fe4440ac150 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.149+0000 7fe44bce8640 1 -- 172.21.15.131:0/340081121 >> 172.21.15.131:0/340081121 conn(0x7fe444069640 msgr2=0x7fe444069a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:04.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.149+0000 7fe44bce8640 1 -- 172.21.15.131:0/340081121 shutdown_connections 2023-09-12T14:55:04.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.149+0000 7fe44bce8640 1 -- 172.21.15.131:0/340081121 wait complete. 2023-09-12T14:55:04.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.153+0000 7fe44bce8640 1 Processor -- start 2023-09-12T14:55:04.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.153+0000 7fe44bce8640 1 -- start start 2023-09-12T14:55:04.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.153+0000 7fe44bce8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe44406ee20 0x7fe4441516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.153+0000 7fe44bce8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4440caab0 0x7fe444154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.153+0000 7fe44bce8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4440ccde0 0x7fe4440b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.153+0000 7fe44bce8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44406c6b0 con 0x7fe44406ee20 2023-09-12T14:55:04.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.153+0000 7fe44bce8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44406baa0 con 0x7fe4440caab0 2023-09-12T14:55:04.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.153+0000 7fe44bce8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe44406bd80 con 0x7fe4440ccde0 2023-09-12T14:55:04.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.153+0000 7fe449a84640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe44406ee20 0x7fe4441516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:04.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.153+0000 7fe449a84640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe44406ee20 0x7fe4441516f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38970/0 (socket says 172.21.15.131:38970) 2023-09-12T14:55:04.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.153+0000 7fe449a84640 1 -- 172.21.15.131:0/1061807060 learned_addr learned my addr 172.21.15.131:0/1061807060 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:04.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.153+0000 7fe44a285640 1 --2- 172.21.15.131:0/1061807060 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4440ccde0 0x7fe4440b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:04.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.153+0000 7fe449283640 1 --2- 172.21.15.131:0/1061807060 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4440caab0 0x7fe444154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:04.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.153+0000 7fe449a84640 1 -- 172.21.15.131:0/1061807060 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4440ccde0 msgr2=0x7fe4440b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.153+0000 7fe449a84640 1 --2- 172.21.15.131:0/1061807060 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4440ccde0 0x7fe4440b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.153+0000 7fe449a84640 1 -- 172.21.15.131:0/1061807060 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4440caab0 msgr2=0x7fe444154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.153+0000 7fe449a84640 1 --2- 172.21.15.131:0/1061807060 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4440caab0 0x7fe444154290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.153+0000 7fe449a84640 1 -- 172.21.15.131:0/1061807060 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4340094a0 con 0x7fe44406ee20 2023-09-12T14:55:04.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.153+0000 7fe449a84640 1 --2- 172.21.15.131:0/1061807060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe44406ee20 0x7fe4441516f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe4340b0040 tx=0x7fe4340a5620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:04.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.153+0000 7fe43a7fc640 1 -- 172.21.15.131:0/1061807060 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe434004230 con 0x7fe44406ee20 2023-09-12T14:55:04.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.153+0000 7fe44bce8640 1 -- 172.21.15.131:0/1061807060 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe44406c060 con 0x7fe44406ee20 2023-09-12T14:55:04.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.153+0000 7fe44bce8640 1 -- 172.21.15.131:0/1061807060 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4440b1230 con 0x7fe44406ee20 2023-09-12T14:55:04.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.157+0000 7fe44bce8640 1 -- 172.21.15.131:0/1061807060 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe44406cb20 con 0x7fe44406ee20 2023-09-12T14:55:04.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.157+0000 7f79fb7fe640 1 -- 172.21.15.131:0/3808573680 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f40ad850 con 0x7f79fc155300 2023-09-12T14:55:04.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.157+0000 7fe43a7fc640 1 -- 172.21.15.131:0/1061807060 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe434004390 con 0x7fe44406ee20 2023-09-12T14:55:04.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.157+0000 7fe43a7fc640 1 -- 172.21.15.131:0/1061807060 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe434003800 con 0x7fe44406ee20 2023-09-12T14:55:04.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.157+0000 7fe43a7fc640 1 -- 172.21.15.131:0/1061807060 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fe4340b4040 con 0x7fe44406ee20 2023-09-12T14:55:04.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.157+0000 7fe43a7fc640 1 --2- 172.21.15.131:0/1061807060 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe428065bd0 0x7fe428068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.157+0000 7fe43a7fc640 1 -- 172.21.15.131:0/1061807060 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe43411d170 con 0x7fe44406ee20 2023-09-12T14:55:04.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.157+0000 7fe449283640 1 --2- 172.21.15.131:0/1061807060 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe428065bd0 0x7fe428068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:04.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f7a02f24640 1 -- 172.21.15.131:0/3808573680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79fc155300 msgr2=0x7f79fc0ac150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f7a02f24640 1 --2- 172.21.15.131:0/3808573680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79fc155300 0x7f79fc0ac150 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f79fc0ccb40 tx=0x7f79f401cd10 comp rx=0 tx=0).stop 2023-09-12T14:55:04.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7fb16d304640 1 -- 172.21.15.131:0/1477148953 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1680ccf20 msgr2=0x7fb1680cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7fb16d304640 1 --2- 172.21.15.131:0/1477148953 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1680ccf20 0x7fb1680cd300 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb1580074e0 tx=0x7fb15801d480 comp rx=0 tx=0).stop 2023-09-12T14:55:04.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7fe449283640 1 --2- 172.21.15.131:0/1061807060 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe428065bd0 0x7fe428068090 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe4440adb90 tx=0x7fe43c001ae0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:04.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f7a02f24640 1 -- 172.21.15.131:0/3808573680 shutdown_connections 2023-09-12T14:55:04.166 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f7a02f24640 1 --2- 172.21.15.131:0/3808573680 >> v2:172.21.15.187:3300/0 conn(0x7f79fc0ccde0 0x7f79fc0cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.166 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f7a02f24640 1 --2- 172.21.15.131:0/3808573680 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79fc15b2e0 0x7f79fc0caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.166 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f7a02f24640 1 --2- 172.21.15.131:0/3808573680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79fc155300 0x7f79fc0ac150 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.166 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f7a02f24640 1 -- 172.21.15.131:0/3808573680 >> 172.21.15.131:0/3808573680 conn(0x7f79fc069640 msgr2=0x7f79fc069a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:04.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7fb16d304640 1 -- 172.21.15.131:0/1477148953 shutdown_connections 2023-09-12T14:55:04.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7fb16d304640 1 --2- 172.21.15.131:0/1477148953 >> v2:172.21.15.187:3300/0 conn(0x7fb1680cc0c0 0x7fb16816b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7fb16d304640 1 --2- 172.21.15.131:0/1477148953 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1680ccf20 0x7fb1680cd300 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb1580074e0 tx=0x7fb15801d480 comp rx=0 tx=0).stop 2023-09-12T14:55:04.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7fb16d304640 1 -- 172.21.15.131:0/1477148953 >> 172.21.15.131:0/1477148953 conn(0x7fb168069640 msgr2=0x7fb168069a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:04.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f7a02f24640 1 -- 172.21.15.131:0/3808573680 shutdown_connections 2023-09-12T14:55:04.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7fb16d304640 1 -- 172.21.15.131:0/1477148953 shutdown_connections 2023-09-12T14:55:04.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f6531b1f640 1 -- 172.21.15.131:0/740600853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f652c140950 msgr2=0x7f652c15b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f6531b1f640 1 --2- 172.21.15.131:0/740600853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f652c140950 0x7f652c15b180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f652c1584d0 tx=0x7f652001d410 comp rx=0 tx=0).stop 2023-09-12T14:55:04.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f6531b1f640 1 -- 172.21.15.131:0/740600853 shutdown_connections 2023-09-12T14:55:04.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f6531b1f640 1 --2- 172.21.15.131:0/740600853 >> v2:172.21.15.131:3300/0 conn(0x7f652c15b6c0 0x7f652c0ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f6531b1f640 1 --2- 172.21.15.131:0/740600853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f652c140950 0x7f652c15b180 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f6531b1f640 1 -- 172.21.15.131:0/740600853 >> 172.21.15.131:0/740600853 conn(0x7f652c065680 msgr2=0x7f652c069240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:04.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f6531b1f640 1 -- 172.21.15.131:0/740600853 shutdown_connections 2023-09-12T14:55:04.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f6531b1f640 1 -- 172.21.15.131:0/740600853 wait complete. 2023-09-12T14:55:04.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f7a02f24640 1 -- 172.21.15.131:0/3808573680 wait complete. 2023-09-12T14:55:04.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f00308c8640 1 -- 172.21.15.131:0/3716959171 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00140047d0 msgr2=0x7f0014004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f00308c8640 1 --2- 172.21.15.131:0/3716959171 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00140047d0 0x7f0014004c50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f002000a9b0 tx=0x7f002001c8a0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f00308c8640 1 -- 172.21.15.131:0/3716959171 shutdown_connections 2023-09-12T14:55:04.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f00308c8640 1 --2- 172.21.15.131:0/3716959171 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00140047d0 0x7f0014004c50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f00308c8640 1 -- 172.21.15.131:0/3716959171 >> 172.21.15.131:0/3716959171 conn(0x7f002c069240 msgr2=0x7f002c069650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:04.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7fb16d304640 1 -- 172.21.15.131:0/1477148953 wait complete. 2023-09-12T14:55:04.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7fb16d304640 1 Processor -- start 2023-09-12T14:55:04.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f00308c8640 1 -- 172.21.15.131:0/3716959171 shutdown_connections 2023-09-12T14:55:04.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f00308c8640 1 -- 172.21.15.131:0/3716959171 wait complete. 2023-09-12T14:55:04.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f7a02f24640 1 Processor -- start 2023-09-12T14:55:04.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f6531b1f640 1 Processor -- start 2023-09-12T14:55:04.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f7a02f24640 1 -- start start 2023-09-12T14:55:04.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f00308c8640 1 Processor -- start 2023-09-12T14:55:04.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f7a02f24640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79fc0ccde0 0x7f79fc14e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f7a02f24640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79fc155300 0x7f79fc0a9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f7a02f24640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79fc15b2e0 0x7f79fc0a9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f7a02f24640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc0a6ff0 con 0x7f79fc155300 2023-09-12T14:55:04.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f7a02f24640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc06bde0 con 0x7f79fc0ccde0 2023-09-12T14:55:04.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f7a02f24640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc06c0c0 con 0x7f79fc15b2e0 2023-09-12T14:55:04.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f79fbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79fc155300 0x7f79fc0a9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:04.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f79fbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79fc155300 0x7f79fc0a9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39018/0 (socket says 172.21.15.131:39018) 2023-09-12T14:55:04.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f79fbfff640 1 -- 172.21.15.131:0/624243696 learned_addr learned my addr 172.21.15.131:0/624243696 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:04.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7f6531b1f640 1 -- start start 2023-09-12T14:55:04.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f7a00cc0640 1 --2- 172.21.15.131:0/624243696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79fc0ccde0 0x7f79fc14e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:04.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f79fbfff640 1 -- 172.21.15.131:0/624243696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79fc15b2e0 msgr2=0x7f79fc0a9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f79fbfff640 1 --2- 172.21.15.131:0/624243696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79fc15b2e0 0x7f79fc0a9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f79fbfff640 1 -- 172.21.15.131:0/624243696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79fc0ccde0 msgr2=0x7f79fc14e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f79fbfff640 1 --2- 172.21.15.131:0/624243696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79fc0ccde0 0x7f79fc14e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f79fbfff640 1 -- 172.21.15.131:0/624243696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f40098e0 con 0x7f79fc155300 2023-09-12T14:55:04.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f6531b1f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f652c140570 0x7f652c152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f6531b1f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f652c140950 0x7f652c1379d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f6531b1f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f652c15b6c0 0x7f652c0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f6531b1f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652c1f8d60 con 0x7f652c15b6c0 2023-09-12T14:55:04.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f6531b1f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652c161470 con 0x7f652c140950 2023-09-12T14:55:04.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f6531b1f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f652c153640 con 0x7f652c140570 2023-09-12T14:55:04.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.161+0000 7fb16d304640 1 -- start start 2023-09-12T14:55:04.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7fb16d304640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1680b0e80 0x7fb1681fb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7fb16d304640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1680cc0c0 0x7fb1681616f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7fb16d304640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb16806bf40 0x7fb16806c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7fb16d304640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1681fc080 con 0x7fb16806bf40 2023-09-12T14:55:04.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7fb16d304640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16806c910 con 0x7fb1680b0e80 2023-09-12T14:55:04.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7fb16d304640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb168161300 con 0x7fb1680cc0c0 2023-09-12T14:55:04.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f79fbfff640 1 --2- 172.21.15.131:0/624243696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79fc155300 0x7f79fc0a9020 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f79f000d750 tx=0x7f79f000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:04.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7fb1677fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb16806bf40 0x7fb16806c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:04.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7fb1677fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb16806bf40 0x7fb16806c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39028/0 (socket says 172.21.15.131:39028) 2023-09-12T14:55:04.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7fb1677fe640 1 -- 172.21.15.131:0/633088984 learned_addr learned my addr 172.21.15.131:0/633088984 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:04.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f652b7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f652c140570 0x7f652c152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:04.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7fb1667fc640 1 --2- 172.21.15.131:0/633088984 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1680cc0c0 0x7fb1681616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:04.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f652bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f652c15b6c0 0x7f652c0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:04.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f652b7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f652c140570 0x7f652c152940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:42712/0 (socket says 172.21.15.131:42712) 2023-09-12T14:55:04.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f652b7fe640 1 -- 172.21.15.131:0/3468936662 learned_addr learned my addr 172.21.15.131:0/3468936662 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:04.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f652affd640 1 --2- 172.21.15.131:0/3468936662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f652c140950 0x7f652c1379d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:04.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7fb1677fe640 1 -- 172.21.15.131:0/633088984 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1680cc0c0 msgr2=0x7fb1681616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.189 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7fb1677fe640 1 --2- 172.21.15.131:0/633088984 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1680cc0c0 0x7fb1681616f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.189 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7fb1677fe640 1 -- 172.21.15.131:0/633088984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1680b0e80 msgr2=0x7fb1681fb200 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-12T14:55:04.189 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7fb1677fe640 1 --2- 172.21.15.131:0/633088984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1680b0e80 0x7fb1681fb200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.189 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7fb1677fe640 1 -- 172.21.15.131:0/633088984 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1580071c0 con 0x7fb16806bf40 2023-09-12T14:55:04.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f652bfff640 1 -- 172.21.15.131:0/3468936662 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f652c140570 msgr2=0x7f652c152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f652bfff640 1 --2- 172.21.15.131:0/3468936662 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f652c140570 0x7f652c152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f652bfff640 1 -- 172.21.15.131:0/3468936662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f652c140950 msgr2=0x7f652c1379d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f652bfff640 1 --2- 172.21.15.131:0/3468936662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f652c140950 0x7f652c1379d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7fb1677fe640 1 --2- 172.21.15.131:0/633088984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb16806bf40 0x7fb16806c3d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb16000cd10 tx=0x7fb16000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:04.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f652bfff640 1 -- 172.21.15.131:0/3468936662 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65200071c0 con 0x7f652c15b6c0 2023-09-12T14:55:04.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f00308c8640 1 -- start start 2023-09-12T14:55:04.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f00308c8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00140047d0 0x7f002c140e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f00308c8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f002c14a5e0 0x7f002c14df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f00308c8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f002c14e4a0 0x7f002c15a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f00308c8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c14e920 con 0x7f002c14a5e0 2023-09-12T14:55:04.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f00308c8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c149330 con 0x7f002c14e4a0 2023-09-12T14:55:04.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f00308c8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002c149540 con 0x7f00140047d0 2023-09-12T14:55:04.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f002a59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f002c14a5e0 0x7f002c14df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:04.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f002a59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f002c14a5e0 0x7f002c14df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39050/0 (socket says 172.21.15.131:39050) 2023-09-12T14:55:04.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f002a59c640 1 -- 172.21.15.131:0/3554426397 learned_addr learned my addr 172.21.15.131:0/3554426397 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:04.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f652bfff640 1 --2- 172.21.15.131:0/3468936662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f652c15b6c0 0x7f652c0aece0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f651c00b3e0 tx=0x7f651c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:04.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f002ad9d640 1 --2- 172.21.15.131:0/3554426397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00140047d0 0x7f002c140e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:04.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f002b59e640 1 --2- 172.21.15.131:0/3554426397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f002c14e4a0 0x7f002c15a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:04.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f002a59c640 1 -- 172.21.15.131:0/3554426397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00140047d0 msgr2=0x7f002c140e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f002a59c640 1 --2- 172.21.15.131:0/3554426397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00140047d0 0x7f002c140e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f002a59c640 1 -- 172.21.15.131:0/3554426397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f002c14e4a0 msgr2=0x7f002c15a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f002a59c640 1 --2- 172.21.15.131:0/3554426397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f002c14e4a0 0x7f002c15a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f002a59c640 1 -- 172.21.15.131:0/3554426397 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c149820 con 0x7f002c14a5e0 2023-09-12T14:55:04.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f002b59e640 1 --2- 172.21.15.131:0/3554426397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f002c14e4a0 0x7f002c15a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:04.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f002ad9d640 1 --2- 172.21.15.131:0/3554426397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00140047d0 0x7f002c140e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:55:04.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f002a59c640 1 --2- 172.21.15.131:0/3554426397 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f002c14a5e0 0x7f002c14df60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f002000a8d0 tx=0x7f00200046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:04.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f0b66ee9640 1 -- 172.21.15.131:0/1844541610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b60069950 msgr2=0x7f0b6015b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f0b66ee9640 1 --2- 172.21.15.131:0/1844541610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b60069950 0x7f0b6015b1a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c009840 tx=0x7f0b5c01c7d0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f79f97fa640 1 -- 172.21.15.131:0/624243696 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f0009c30 con 0x7f79fc155300 2023-09-12T14:55:04.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7fe43a7fc640 1 -- 172.21.15.131:0/1061807060 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fe4340eabe0 con 0x7fe44406ee20 2023-09-12T14:55:04.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f0b66ee9640 1 -- 172.21.15.131:0/1844541610 shutdown_connections 2023-09-12T14:55:04.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f0b66ee9640 1 --2- 172.21.15.131:0/1844541610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b60069950 0x7f0b6015b1a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f0b66ee9640 1 -- 172.21.15.131:0/1844541610 >> 172.21.15.131:0/1844541610 conn(0x7f0b6006e170 msgr2=0x7f0b6006a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:04.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f0b66ee9640 1 -- 172.21.15.131:0/1844541610 shutdown_connections 2023-09-12T14:55:04.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.165+0000 7f7a02f24640 1 -- 172.21.15.131:0/624243696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79fc06c3a0 con 0x7f79fc155300 2023-09-12T14:55:04.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f7a02f24640 1 -- 172.21.15.131:0/624243696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79fc06b0b0 con 0x7f79fc155300 2023-09-12T14:55:04.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f6513fff640 1 -- 172.21.15.131:0/3468936662 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c011830 con 0x7f652c15b6c0 2023-09-12T14:55:04.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f6513fff640 1 -- 172.21.15.131:0/3468936662 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f651c011aa0 con 0x7f652c15b6c0 2023-09-12T14:55:04.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f6513fff640 1 -- 172.21.15.131:0/3468936662 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c010380 con 0x7f652c15b6c0 2023-09-12T14:55:04.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f00137fe640 1 -- 172.21.15.131:0/3554426397 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00200a5e80 con 0x7f002c14a5e0 2023-09-12T14:55:04.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f6531b1f640 1 -- 172.21.15.131:0/3468936662 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f652c161710 con 0x7f652c15b6c0 2023-09-12T14:55:04.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f6531b1f640 1 -- 172.21.15.131:0/3468936662 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f652c1538e0 con 0x7f652c15b6c0 2023-09-12T14:55:04.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f00308c8640 1 -- 172.21.15.131:0/3554426397 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002c149b00 con 0x7f002c14a5e0 2023-09-12T14:55:04.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f00308c8640 1 -- 172.21.15.131:0/3554426397 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f002c14a0c0 con 0x7f002c14a5e0 2023-09-12T14:55:04.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f00308c8640 1 -- 172.21.15.131:0/3554426397 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f002c04e110 con 0x7f002c14a5e0 2023-09-12T14:55:04.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f00137fe640 1 -- 172.21.15.131:0/3554426397 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f00200b87b0 con 0x7f002c14a5e0 2023-09-12T14:55:04.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f79f97fa640 1 -- 172.21.15.131:0/624243696 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f79f0009e80 con 0x7f79fc155300 2023-09-12T14:55:04.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f79f97fa640 1 -- 172.21.15.131:0/624243696 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f0004260 con 0x7f79fc155300 2023-09-12T14:55:04.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f79f97fa640 1 -- 172.21.15.131:0/624243696 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f79f0005020 con 0x7f79fc155300 2023-09-12T14:55:04.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f64f1344640 1 -- 172.21.15.131:0/289464284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64e4054140 msgr2=0x7f64e40e31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f64f1344640 1 --2- 172.21.15.131:0/289464284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64e4054140 0x7f64e40e31f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0074e0 tx=0x7f64dc01d4e0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f64f1344640 1 -- 172.21.15.131:0/289464284 shutdown_connections 2023-09-12T14:55:04.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f64f1344640 1 --2- 172.21.15.131:0/289464284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64e4054140 0x7f64e40e31f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f64f1344640 1 --2- 172.21.15.131:0/289464284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64e4053ce0 0x7f64e40e2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f64f1344640 1 -- 172.21.15.131:0/289464284 >> 172.21.15.131:0/289464284 conn(0x7f64e4054b10 msgr2=0x7f64e4054f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:04.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f64f1344640 1 -- 172.21.15.131:0/289464284 shutdown_connections 2023-09-12T14:55:04.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f0b66ee9640 1 -- 172.21.15.131:0/1844541610 wait complete. 2023-09-12T14:55:04.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f0b66ee9640 1 Processor -- start 2023-09-12T14:55:04.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f00137fe640 1 -- 172.21.15.131:0/3554426397 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00200b8b40 con 0x7f002c14a5e0 2023-09-12T14:55:04.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f79f97fa640 1 --2- 172.21.15.131:0/624243696 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f79e8065b20 0x7f79e8067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f79f97fa640 1 -- 172.21.15.131:0/624243696 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79f00834d0 con 0x7f79fc155300 2023-09-12T14:55:04.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f0b66ee9640 1 -- start start 2023-09-12T14:55:04.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f0b66ee9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b60069950 0x7f0b60203910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f0b66ee9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b60154fd0 0x7f0b60200a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f0b66ee9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b601553b0 0x7f0b6006fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f0b66ee9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b60151ae0 con 0x7f0b601553b0 2023-09-12T14:55:04.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f0b66ee9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b60203e50 con 0x7f0b60069950 2023-09-12T14:55:04.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f0b66ee9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b60151720 con 0x7f0b60154fd0 2023-09-12T14:55:04.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f7a00cc0640 1 --2- 172.21.15.131:0/624243696 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f79e8065b20 0x7f79e8067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:04.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f0b656e6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b60154fd0 0x7f0b60200a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:04.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f0b656e6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b60154fd0 0x7f0b60200a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:42730/0 (socket says 172.21.15.131:42730) 2023-09-12T14:55:04.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f0b656e6640 1 -- 172.21.15.131:0/4278997302 learned_addr learned my addr 172.21.15.131:0/4278997302 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:04.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f0b65ee7640 1 --2- 172.21.15.131:0/4278997302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b60069950 0x7f0b60203910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:04.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.169+0000 7f64f1344640 1 -- 172.21.15.131:0/289464284 wait complete. 2023-09-12T14:55:04.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f64f1344640 1 Processor -- start 2023-09-12T14:55:04.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7fb14f7fe640 1 -- 172.21.15.131:0/633088984 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16000f450 con 0x7fb16806bf40 2023-09-12T14:55:04.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f7a00cc0640 1 --2- 172.21.15.131:0/624243696 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f79e8065b20 0x7f79e8067fe0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f79f40af000 tx=0x7f79f401b550 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:04.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7fb14f7fe640 1 -- 172.21.15.131:0/633088984 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb16000f6a0 con 0x7fb16806bf40 2023-09-12T14:55:04.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f0b656e6640 1 -- 172.21.15.131:0/4278997302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b60069950 msgr2=0x7f0b60203910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f0b656e6640 1 --2- 172.21.15.131:0/4278997302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b60069950 0x7f0b60203910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f0b656e6640 1 -- 172.21.15.131:0/4278997302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b601553b0 msgr2=0x7f0b6006fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f0b656e6640 1 --2- 172.21.15.131:0/4278997302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b601553b0 0x7f0b6006fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f0b656e6640 1 -- 172.21.15.131:0/4278997302 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5c0094a0 con 0x7f0b60154fd0 2023-09-12T14:55:04.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f0b656e6640 1 --2- 172.21.15.131:0/4278997302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b60154fd0 0x7f0b60200a10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0b5000c3d0 tx=0x7f0b5000c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:04.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f0b567fc640 1 -- 172.21.15.131:0/4278997302 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b50011790 con 0x7f0b60154fd0 2023-09-12T14:55:04.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f00137fe640 1 -- 172.21.15.131:0/3554426397 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f00200b8ca0 con 0x7f002c14a5e0 2023-09-12T14:55:04.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f0b567fc640 1 -- 172.21.15.131:0/4278997302 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0b500119e0 con 0x7f0b60154fd0 2023-09-12T14:55:04.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f0b567fc640 1 -- 172.21.15.131:0/4278997302 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b50011e10 con 0x7f0b60154fd0 2023-09-12T14:55:04.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f0b66ee9640 1 -- 172.21.15.131:0/4278997302 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b602014d0 con 0x7f0b60154fd0 2023-09-12T14:55:04.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f0b66ee9640 1 -- 172.21.15.131:0/4278997302 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b60153be0 con 0x7f0b60154fd0 2023-09-12T14:55:04.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7fb16d304640 1 -- 172.21.15.131:0/633088984 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1681fb7e0 con 0x7fb16806bf40 2023-09-12T14:55:04.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7fb16d304640 1 -- 172.21.15.131:0/633088984 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1681fbda0 con 0x7fb16806bf40 2023-09-12T14:55:04.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f00137fe640 1 --2- 172.21.15.131:0/3554426397 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0018065c20 0x7f00180680e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7fb14f7fe640 1 -- 172.21.15.131:0/633088984 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16000fa90 con 0x7fb16806bf40 2023-09-12T14:55:04.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f002ad9d640 1 --2- 172.21.15.131:0/3554426397 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0018065c20 0x7f00180680e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:04.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f6513fff640 1 -- 172.21.15.131:0/3468936662 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f651c0104e0 con 0x7f652c15b6c0 2023-09-12T14:55:04.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f6513fff640 1 --2- 172.21.15.131:0/3468936662 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6518065ba0 0x7f6518068060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f6513fff640 1 -- 172.21.15.131:0/3468936662 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f651c083440 con 0x7f652c15b6c0 2023-09-12T14:55:04.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f652b7fe640 1 --2- 172.21.15.131:0/3468936662 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6518065ba0 0x7f6518068060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:04.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f6531b1f640 1 -- 172.21.15.131:0/3468936662 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f652c04e110 con 0x7f652c15b6c0 2023-09-12T14:55:04.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f002ad9d640 1 --2- 172.21.15.131:0/3554426397 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0018065c20 0x7f00180680e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f001c005d10 tx=0x7f001c00b690 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:04.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f00137fe640 1 -- 172.21.15.131:0/3554426397 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0020122260 con 0x7f002c14a5e0 2023-09-12T14:55:04.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f64f1344640 1 -- start start 2023-09-12T14:55:04.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f7a02f24640 1 -- 172.21.15.131:0/624243696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79dc005260 con 0x7f79fc155300 2023-09-12T14:55:04.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f64f1344640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64e4053ce0 0x7f64e4035700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f64f1344640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64e4054140 0x7f64e4033cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f64f1344640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64e40df2f0 0x7f64e40e03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f64f1344640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e4035270 con 0x7f64e4054140 2023-09-12T14:55:04.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f64f1344640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e4034660 con 0x7f64e40df2f0 2023-09-12T14:55:04.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f64f1344640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e4034940 con 0x7f64e4053ce0 2023-09-12T14:55:04.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f652b7fe640 1 --2- 172.21.15.131:0/3468936662 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6518065ba0 0x7f6518068060 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f652c159cd0 tx=0x7f6524003e70 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:04.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f64eaffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64e4053ce0 0x7f64e4035700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:04.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f64ea7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64e4054140 0x7f64e4033cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:04.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f64ea7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64e4054140 0x7f64e4033cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39060/0 (socket says 172.21.15.131:39060) 2023-09-12T14:55:04.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7f64ea7fc640 1 -- 172.21.15.131:0/3812485072 learned_addr learned my addr 172.21.15.131:0/3812485072 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:04.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.173+0000 7fb16d304640 1 -- 172.21.15.131:0/633088984 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb16804e110 con 0x7fb16806bf40 2023-09-12T14:55:04.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.177+0000 7f64ea7fc640 1 -- 172.21.15.131:0/3812485072 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64e4053ce0 msgr2=0x7f64e4035700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.177+0000 7f64ea7fc640 1 --2- 172.21.15.131:0/3812485072 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64e4053ce0 0x7f64e4035700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.177+0000 7f64ea7fc640 1 -- 172.21.15.131:0/3812485072 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64e40df2f0 msgr2=0x7f64e40e03a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-12T14:55:04.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.177+0000 7f64ea7fc640 1 --2- 172.21.15.131:0/3812485072 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64e40df2f0 0x7f64e40e03a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.177+0000 7f64ea7fc640 1 -- 172.21.15.131:0/3812485072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64dc0071c0 con 0x7f64e4054140 2023-09-12T14:55:04.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.177+0000 7f64ea7fc640 1 --2- 172.21.15.131:0/3812485072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64e4054140 0x7f64e4033cf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f64e000d980 tx=0x7f64e000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:04.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.177+0000 7f0b567fc640 1 -- 172.21.15.131:0/4278997302 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f0b5001c460 con 0x7f0b60154fd0 2023-09-12T14:55:04.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.177+0000 7f0b567fc640 1 --2- 172.21.15.131:0/4278997302 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0b44065b20 0x7f0b44067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.177+0000 7f0b65ee7640 1 --2- 172.21.15.131:0/4278997302 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0b44065b20 0x7f0b44067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:04.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.177+0000 7f0b567fc640 1 -- 172.21.15.131:0/4278997302 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b500833c0 con 0x7f0b60154fd0 2023-09-12T14:55:04.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.177+0000 7f0b65ee7640 1 --2- 172.21.15.131:0/4278997302 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0b44065b20 0x7f0b44067fe0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0b580012f0 tx=0x7f0b58009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:04.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.177+0000 7f00137fe640 1 -- 172.21.15.131:0/3554426397 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f00200efd00 con 0x7f002c14a5e0 2023-09-12T14:55:04.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.181+0000 7f0b66ee9640 1 -- 172.21.15.131:0/4278997302 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b48005260 con 0x7f0b60154fd0 2023-09-12T14:55:04.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.181+0000 7fb14f7fe640 1 -- 172.21.15.131:0/633088984 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fb160018400 con 0x7fb16806bf40 2023-09-12T14:55:04.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.181+0000 7fb14f7fe640 1 --2- 172.21.15.131:0/633088984 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb154065b70 0x7fb154068030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.181+0000 7f6513fff640 1 -- 172.21.15.131:0/3468936662 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f651c050e80 con 0x7f652c15b6c0 2023-09-12T14:55:04.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.181+0000 7f64d37fe640 1 -- 172.21.15.131:0/3812485072 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0009c30 con 0x7f64e4054140 2023-09-12T14:55:04.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.181+0000 7f64f1344640 1 -- 172.21.15.131:0/3812485072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e4034c20 con 0x7f64e4054140 2023-09-12T14:55:04.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.185+0000 7f64f1344640 1 -- 172.21.15.131:0/3812485072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e4035c40 con 0x7f64e4054140 2023-09-12T14:55:04.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.185+0000 7fb166ffd640 1 --2- 172.21.15.131:0/633088984 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb154065b70 0x7fb154068030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:04.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.185+0000 7fb166ffd640 1 --2- 172.21.15.131:0/633088984 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb154065b70 0x7fb154068030 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb1580ae000 tx=0x7fb15801bcc0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:04.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.185+0000 7f64f1344640 1 -- 172.21.15.131:0/3812485072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e4005350 con 0x7f64e4054140 2023-09-12T14:55:04.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.189+0000 7f79f97fa640 1 -- 172.21.15.131:0/624243696 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f79f0017080 con 0x7f79fc155300 2023-09-12T14:55:04.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.189+0000 7fb14f7fe640 1 -- 172.21.15.131:0/633088984 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb160056010 con 0x7fb16806bf40 2023-09-12T14:55:04.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.189+0000 7f64d37fe640 1 -- 172.21.15.131:0/3812485072 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f64e000b5e0 con 0x7f64e4054140 2023-09-12T14:55:04.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.189+0000 7f64d37fe640 1 -- 172.21.15.131:0/3812485072 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0004020 con 0x7f64e4054140 2023-09-12T14:55:04.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.193+0000 7f0b567fc640 1 -- 172.21.15.131:0/4278997302 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f0b50050c80 con 0x7f0b60154fd0 2023-09-12T14:55:04.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.201+0000 7fb14f7fe640 1 -- 172.21.15.131:0/633088984 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fb160051c50 con 0x7fb16806bf40 2023-09-12T14:55:04.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.201+0000 7f01c8d41640 1 -- 172.21.15.131:0/173640548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01c40b2590 msgr2=0x7f01c406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.201+0000 7f01c27fc640 1 -- 172.21.15.131:0/173640548 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b80ad440 con 0x7f01c40b2590 2023-09-12T14:55:04.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.201+0000 7f01c8d41640 1 --2- 172.21.15.131:0/173640548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01c40b2590 0x7f01c406d420 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f01c40b04d0 tx=0x7f01b801c870 comp rx=0 tx=0).stop 2023-09-12T14:55:04.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.201+0000 7f01c8d41640 1 -- 172.21.15.131:0/173640548 shutdown_connections 2023-09-12T14:55:04.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.201+0000 7f01c8d41640 1 --2- 172.21.15.131:0/173640548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01c40b2590 0x7f01c406d420 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.201+0000 7f01c8d41640 1 --2- 172.21.15.131:0/173640548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01c40b15d0 0x7f01c406cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.201+0000 7f01c8d41640 1 --2- 172.21.15.131:0/173640548 >> v2:172.21.15.187:3300/0 conn(0x7f01c41579e0 0x7f01c40b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.201+0000 7f01c8d41640 1 -- 172.21.15.131:0/173640548 >> 172.21.15.131:0/173640548 conn(0x7f01c40cd850 msgr2=0x7f01c40cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:04.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.201+0000 7f01c8d41640 1 -- 172.21.15.131:0/173640548 shutdown_connections 2023-09-12T14:55:04.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.201+0000 7f64d37fe640 1 -- 172.21.15.131:0/3812485072 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f64e0004180 con 0x7f64e4054140 2023-09-12T14:55:04.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.201+0000 7f64d37fe640 1 --2- 172.21.15.131:0/3812485072 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f64d8065b20 0x7f64d8067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.201+0000 7f64d37fe640 1 -- 172.21.15.131:0/3812485072 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64e00838c0 con 0x7f64e4054140 2023-09-12T14:55:04.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.201+0000 7f64eaffd640 1 --2- 172.21.15.131:0/3812485072 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f64d8065b20 0x7f64d8067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:04.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.201+0000 7f64eaffd640 1 --2- 172.21.15.131:0/3812485072 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f64d8065b20 0x7f64d8067fe0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f64ec068a70 tx=0x7f64ec08a040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:04.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.205+0000 7f01c8d41640 1 -- 172.21.15.131:0/173640548 wait complete. 2023-09-12T14:55:04.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.205+0000 7f01c8d41640 1 Processor -- start 2023-09-12T14:55:04.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.205+0000 7f01c8d41640 1 -- start start 2023-09-12T14:55:04.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.205+0000 7f01c8d41640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01c40b15d0 0x7f01c4168320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.205+0000 7f01c8d41640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01c40b2590 0x7f01c41536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.205+0000 7f01c8d41640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01c41579e0 0x7f01c4166290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.205+0000 7f01c2ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01c40b2590 0x7f01c41536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:04.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.205+0000 7f01c2ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01c40b2590 0x7f01c41536a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39076/0 (socket says 172.21.15.131:39076) 2023-09-12T14:55:04.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.205+0000 7f01c2ffd640 1 -- 172.21.15.131:0/3568277971 learned_addr learned my addr 172.21.15.131:0/3568277971 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:04.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.205+0000 7f01c37fe640 1 --2- 172.21.15.131:0/3568277971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01c40b15d0 0x7f01c4168320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:04.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.205+0000 7f01c8d41640 1 -- 172.21.15.131:0/3568277971 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c4168860 con 0x7f01c40b2590 2023-09-12T14:55:04.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.205+0000 7f01c8d41640 1 -- 172.21.15.131:0/3568277971 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c4153be0 con 0x7f01c41579e0 2023-09-12T14:55:04.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.205+0000 7f01c8d41640 1 -- 172.21.15.131:0/3568277971 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c4153210 con 0x7f01c40b15d0 2023-09-12T14:55:04.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.209+0000 7f01c2ffd640 1 -- 172.21.15.131:0/3568277971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01c40b15d0 msgr2=0x7f01c4168320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.209+0000 7f01c2ffd640 1 --2- 172.21.15.131:0/3568277971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01c40b15d0 0x7f01c4168320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.209+0000 7f01c2ffd640 1 -- 172.21.15.131:0/3568277971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01c41579e0 msgr2=0x7f01c4166290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.209+0000 7f01c2ffd640 1 --2- 172.21.15.131:0/3568277971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01c41579e0 0x7f01c4166290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.209+0000 7f01c2ffd640 1 -- 172.21.15.131:0/3568277971 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b8009560 con 0x7f01c40b2590 2023-09-12T14:55:04.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.209+0000 7f01c2ffd640 1 --2- 172.21.15.131:0/3568277971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01c40b2590 0x7f01c41536a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f01b400c970 tx=0x7f01b400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:04.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.209+0000 7f64d37fe640 1 -- 172.21.15.131:0/3812485072 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f64e00512a0 con 0x7f64e4054140 2023-09-12T14:55:04.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.209+0000 7f01abfff640 1 -- 172.21.15.131:0/3568277971 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b4007bb0 con 0x7f01c40b2590 2023-09-12T14:55:04.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.209+0000 7f01c8d41640 1 -- 172.21.15.131:0/3568277971 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c4152c30 con 0x7f01c40b2590 2023-09-12T14:55:04.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.209+0000 7f01c8d41640 1 -- 172.21.15.131:0/3568277971 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c4166820 con 0x7f01c40b2590 2023-09-12T14:55:04.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.209+0000 7f01abfff640 1 -- 172.21.15.131:0/3568277971 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f01b4007e00 con 0x7f01c40b2590 2023-09-12T14:55:04.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.209+0000 7f01abfff640 1 -- 172.21.15.131:0/3568277971 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b4005280 con 0x7f01c40b2590 2023-09-12T14:55:04.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.213+0000 7f01abfff640 1 -- 172.21.15.131:0/3568277971 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f01b40053e0 con 0x7f01c40b2590 2023-09-12T14:55:04.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.213+0000 7f01abfff640 1 --2- 172.21.15.131:0/3568277971 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f01ac065ce0 0x7f01ac0681a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:04.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.213+0000 7f01c37fe640 1 --2- 172.21.15.131:0/3568277971 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f01ac065ce0 0x7f01ac0681a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:04.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.213+0000 7f01c37fe640 1 --2- 172.21.15.131:0/3568277971 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f01ac065ce0 0x7f01ac0681a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00b480 tx=0x7f01bc00d040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:04.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.213+0000 7f01abfff640 1 -- 172.21.15.131:0/3568277971 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01b40836d0 con 0x7f01c40b2590 2023-09-12T14:55:04.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.217+0000 7f01c8d41640 1 -- 172.21.15.131:0/3568277971 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01c4153380 con 0x7f01c40b2590 2023-09-12T14:55:04.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.225+0000 7f01abfff640 1 -- 172.21.15.131:0/3568277971 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f01b40511b0 con 0x7f01c40b2590 2023-09-12T14:55:04.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.477+0000 7fe44bce8640 1 -- 172.21.15.131:0/1061807060 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe44404e110 con 0x7fe44406ee20 2023-09-12T14:55:04.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.481+0000 7fe43a7fc640 1 -- 172.21.15.131:0/1061807060 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe4340ee200 con 0x7fe44406ee20 2023-09-12T14:55:04.484 INFO:teuthology.orchestra.run.smithi131.stdout:55834574853 2023-09-12T14:55:04.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.481+0000 7fe427fff640 1 -- 172.21.15.131:0/1061807060 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe428065bd0 msgr2=0x7fe428068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.481+0000 7fe427fff640 1 --2- 172.21.15.131:0/1061807060 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe428065bd0 0x7fe428068090 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe4440adb90 tx=0x7fe43c001ae0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.481+0000 7fe427fff640 1 -- 172.21.15.131:0/1061807060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe44406ee20 msgr2=0x7fe4441516f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.481+0000 7fe427fff640 1 --2- 172.21.15.131:0/1061807060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe44406ee20 0x7fe4441516f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe4340b0040 tx=0x7fe4340a5620 comp rx=0 tx=0).stop 2023-09-12T14:55:04.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.481+0000 7fe427fff640 1 -- 172.21.15.131:0/1061807060 shutdown_connections 2023-09-12T14:55:04.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.481+0000 7fe427fff640 1 --2- 172.21.15.131:0/1061807060 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe428065bd0 0x7fe428068090 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.481+0000 7fe427fff640 1 --2- 172.21.15.131:0/1061807060 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4440ccde0 0x7fe4440b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.481+0000 7fe427fff640 1 --2- 172.21.15.131:0/1061807060 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4440caab0 0x7fe444154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.481+0000 7fe427fff640 1 --2- 172.21.15.131:0/1061807060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe44406ee20 0x7fe4441516f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.481+0000 7fe427fff640 1 -- 172.21.15.131:0/1061807060 >> 172.21.15.131:0/1061807060 conn(0x7fe444069640 msgr2=0x7fe4440acc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:04.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.481+0000 7fe427fff640 1 -- 172.21.15.131:0/1061807060 shutdown_connections 2023-09-12T14:55:04.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.481+0000 7fe427fff640 1 -- 172.21.15.131:0/1061807060 wait complete. 2023-09-12T14:55:04.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.489+0000 7f00308c8640 1 -- 172.21.15.131:0/3554426397 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f002c0c8f20 con 0x7f002c14a5e0 2023-09-12T14:55:04.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.489+0000 7f00137fe640 1 -- 172.21.15.131:0/3554426397 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f00200f3320 con 0x7f002c14a5e0 2023-09-12T14:55:04.495 INFO:teuthology.orchestra.run.smithi131.stdout:55834574853 2023-09-12T14:55:04.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.493+0000 7f7a02f24640 1 -- 172.21.15.131:0/624243696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f79dc0050d0 con 0x7f79fc155300 2023-09-12T14:55:04.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.493+0000 7f79f97fa640 1 -- 172.21.15.131:0/624243696 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f79f0050eb0 con 0x7f79fc155300 2023-09-12T14:55:04.497 INFO:teuthology.orchestra.run.smithi131.stdout:55834574853 2023-09-12T14:55:04.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.497+0000 7f7a02f24640 1 -- 172.21.15.131:0/624243696 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f79e8065b20 msgr2=0x7f79e8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.497+0000 7f00308c8640 1 -- 172.21.15.131:0/3554426397 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0018065c20 msgr2=0x7f00180680e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.497+0000 7f7a02f24640 1 --2- 172.21.15.131:0/624243696 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f79e8065b20 0x7f79e8067fe0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f79f40af000 tx=0x7f79f401b550 comp rx=0 tx=0).stop 2023-09-12T14:55:04.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.497+0000 7f7a02f24640 1 -- 172.21.15.131:0/624243696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79fc155300 msgr2=0x7f79fc0a9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.497+0000 7f7a02f24640 1 --2- 172.21.15.131:0/624243696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79fc155300 0x7f79fc0a9020 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f79f000d750 tx=0x7f79f000dc10 comp rx=0 tx=0).stop 2023-09-12T14:55:04.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.497+0000 7f00308c8640 1 --2- 172.21.15.131:0/3554426397 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0018065c20 0x7f00180680e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f001c005d10 tx=0x7f001c00b690 comp rx=0 tx=0).stop 2023-09-12T14:55:04.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.497+0000 7f00308c8640 1 -- 172.21.15.131:0/3554426397 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f002c14a5e0 msgr2=0x7f002c14df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.497+0000 7f00308c8640 1 --2- 172.21.15.131:0/3554426397 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f002c14a5e0 0x7f002c14df60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f002000a8d0 tx=0x7f00200046d0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.497+0000 7f00308c8640 1 -- 172.21.15.131:0/3554426397 shutdown_connections 2023-09-12T14:55:04.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.497+0000 7f00308c8640 1 --2- 172.21.15.131:0/3554426397 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0018065c20 0x7f00180680e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.497+0000 7f00308c8640 1 --2- 172.21.15.131:0/3554426397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f002c14e4a0 0x7f002c15a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.497+0000 7f00308c8640 1 --2- 172.21.15.131:0/3554426397 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f002c14a5e0 0x7f002c14df60 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f002000a8d0 tx=0x7f00200046d0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.497+0000 7f00308c8640 1 --2- 172.21.15.131:0/3554426397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00140047d0 0x7f002c140e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.497+0000 7f00308c8640 1 -- 172.21.15.131:0/3554426397 >> 172.21.15.131:0/3554426397 conn(0x7f002c069240 msgr2=0x7f002c0af370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:04.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.497+0000 7f7a02f24640 1 -- 172.21.15.131:0/624243696 shutdown_connections 2023-09-12T14:55:04.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.497+0000 7f7a02f24640 1 --2- 172.21.15.131:0/624243696 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f79e8065b20 0x7f79e8067fe0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.497+0000 7f00308c8640 1 -- 172.21.15.131:0/3554426397 shutdown_connections 2023-09-12T14:55:04.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.497+0000 7f00308c8640 1 -- 172.21.15.131:0/3554426397 wait complete. 2023-09-12T14:55:04.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.497+0000 7f7a02f24640 1 --2- 172.21.15.131:0/624243696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79fc15b2e0 0x7f79fc0a9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.497+0000 7f7a02f24640 1 --2- 172.21.15.131:0/624243696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79fc155300 0x7f79fc0a9020 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.497+0000 7f7a02f24640 1 --2- 172.21.15.131:0/624243696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79fc0ccde0 0x7f79fc14e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.497+0000 7f7a02f24640 1 -- 172.21.15.131:0/624243696 >> 172.21.15.131:0/624243696 conn(0x7f79fc069640 msgr2=0x7f79fc0b18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:04.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.497+0000 7f7a02f24640 1 -- 172.21.15.131:0/624243696 shutdown_connections 2023-09-12T14:55:04.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.497+0000 7f7a02f24640 1 -- 172.21.15.131:0/624243696 wait complete. 2023-09-12T14:55:04.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.501+0000 7f0b66ee9640 1 -- 172.21.15.131:0/4278997302 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0b48005510 con 0x7f0b60154fd0 2023-09-12T14:55:04.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.501+0000 7f0b567fc640 1 -- 172.21.15.131:0/4278997302 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0b500542a0 con 0x7f0b60154fd0 2023-09-12T14:55:04.506 INFO:teuthology.orchestra.run.smithi131.stdout:55834574852 2023-09-12T14:55:04.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.505+0000 7f0b43fff640 1 -- 172.21.15.131:0/4278997302 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0b44065b20 msgr2=0x7f0b44067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.505+0000 7f0b43fff640 1 --2- 172.21.15.131:0/4278997302 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0b44065b20 0x7f0b44067fe0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0b580012f0 tx=0x7f0b58009040 comp rx=0 tx=0).stop 2023-09-12T14:55:04.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.505+0000 7f0b43fff640 1 -- 172.21.15.131:0/4278997302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b60154fd0 msgr2=0x7f0b60200a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.505+0000 7f0b43fff640 1 --2- 172.21.15.131:0/4278997302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b60154fd0 0x7f0b60200a10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0b5000c3d0 tx=0x7f0b5000c890 comp rx=0 tx=0).stop 2023-09-12T14:55:04.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.505+0000 7f6531b1f640 1 -- 172.21.15.131:0/3468936662 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f652c06b2e0 con 0x7f652c15b6c0 2023-09-12T14:55:04.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.505+0000 7f0b43fff640 1 -- 172.21.15.131:0/4278997302 shutdown_connections 2023-09-12T14:55:04.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.505+0000 7f0b43fff640 1 --2- 172.21.15.131:0/4278997302 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0b44065b20 0x7f0b44067fe0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.505+0000 7f0b43fff640 1 --2- 172.21.15.131:0/4278997302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b60154fd0 0x7f0b60200a10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.505+0000 7f0b43fff640 1 --2- 172.21.15.131:0/4278997302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b601553b0 0x7f0b6006fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.505+0000 7f0b43fff640 1 --2- 172.21.15.131:0/4278997302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b60069950 0x7f0b60203910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.505+0000 7f0b43fff640 1 -- 172.21.15.131:0/4278997302 >> 172.21.15.131:0/4278997302 conn(0x7f0b6006e170 msgr2=0x7f0b600ca210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:04.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.505+0000 7f6513fff640 1 -- 172.21.15.131:0/3468936662 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f651c015070 con 0x7f652c15b6c0 2023-09-12T14:55:04.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.505+0000 7f0b43fff640 1 -- 172.21.15.131:0/4278997302 shutdown_connections 2023-09-12T14:55:04.512 INFO:teuthology.orchestra.run.smithi131.stdout:55834574853 2023-09-12T14:55:04.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.505+0000 7f0b43fff640 1 -- 172.21.15.131:0/4278997302 wait complete. 2023-09-12T14:55:04.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.509+0000 7fb16d304640 1 -- 172.21.15.131:0/633088984 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb1680cb920 con 0x7fb16806bf40 2023-09-12T14:55:04.513 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2023-09-12T14:55:04.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.509+0000 7f6531b1f640 1 -- 172.21.15.131:0/3468936662 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6518065ba0 msgr2=0x7f6518068060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.509+0000 7f6531b1f640 1 --2- 172.21.15.131:0/3468936662 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6518065ba0 0x7f6518068060 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f652c159cd0 tx=0x7f6524003e70 comp rx=0 tx=0).stop 2023-09-12T14:55:04.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.509+0000 7f6531b1f640 1 -- 172.21.15.131:0/3468936662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f652c15b6c0 msgr2=0x7f652c0aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.509+0000 7f6531b1f640 1 --2- 172.21.15.131:0/3468936662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f652c15b6c0 0x7f652c0aece0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f651c00b3e0 tx=0x7f651c00b8a0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.509+0000 7fb14f7fe640 1 -- 172.21.15.131:0/633088984 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb160055270 con 0x7fb16806bf40 2023-09-12T14:55:04.515 INFO:teuthology.orchestra.run.smithi131.stdout:55834574853 2023-09-12T14:55:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.509+0000 7f6531b1f640 1 -- 172.21.15.131:0/3468936662 shutdown_connections 2023-09-12T14:55:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.509+0000 7f6531b1f640 1 --2- 172.21.15.131:0/3468936662 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6518065ba0 0x7f6518068060 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.509+0000 7f6531b1f640 1 --2- 172.21.15.131:0/3468936662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f652c15b6c0 0x7f652c0aece0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.509+0000 7f6531b1f640 1 --2- 172.21.15.131:0/3468936662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f652c140950 0x7f652c1379d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.509+0000 7f6531b1f640 1 --2- 172.21.15.131:0/3468936662 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f652c140570 0x7f652c152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.509+0000 7f6531b1f640 1 -- 172.21.15.131:0/3468936662 >> 172.21.15.131:0/3468936662 conn(0x7f652c065680 msgr2=0x7f652c157390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:04.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.509+0000 7f6531b1f640 1 -- 172.21.15.131:0/3468936662 shutdown_connections 2023-09-12T14:55:04.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.509+0000 7f6531b1f640 1 -- 172.21.15.131:0/3468936662 wait complete. 2023-09-12T14:55:04.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.513+0000 7fb16d304640 1 -- 172.21.15.131:0/633088984 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb154065b70 msgr2=0x7fb154068030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.513+0000 7fb16d304640 1 --2- 172.21.15.131:0/633088984 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb154065b70 0x7fb154068030 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb1580ae000 tx=0x7fb15801bcc0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.513+0000 7fb16d304640 1 -- 172.21.15.131:0/633088984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb16806bf40 msgr2=0x7fb16806c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.513+0000 7fb16d304640 1 --2- 172.21.15.131:0/633088984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb16806bf40 0x7fb16806c3d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb16000cd10 tx=0x7fb16000a620 comp rx=0 tx=0).stop 2023-09-12T14:55:04.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.513+0000 7fb16d304640 1 -- 172.21.15.131:0/633088984 shutdown_connections 2023-09-12T14:55:04.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.513+0000 7fb16d304640 1 --2- 172.21.15.131:0/633088984 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb154065b70 0x7fb154068030 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.513+0000 7fb16d304640 1 --2- 172.21.15.131:0/633088984 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1680cc0c0 0x7fb1681616f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.513+0000 7fb16d304640 1 --2- 172.21.15.131:0/633088984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb16806bf40 0x7fb16806c3d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.513+0000 7fb16d304640 1 --2- 172.21.15.131:0/633088984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1680b0e80 0x7fb1681fb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.513+0000 7fb16d304640 1 -- 172.21.15.131:0/633088984 >> 172.21.15.131:0/633088984 conn(0x7fb168069640 msgr2=0x7fb1681faa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:04.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.513+0000 7fb16d304640 1 -- 172.21.15.131:0/633088984 shutdown_connections 2023-09-12T14:55:04.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.513+0000 7fb16d304640 1 -- 172.21.15.131:0/633088984 wait complete. 2023-09-12T14:55:04.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.513+0000 7f64f1344640 1 -- 172.21.15.131:0/3812485072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f64e4052c50 con 0x7f64e4054140 2023-09-12T14:55:04.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.513+0000 7f64d37fe640 1 -- 172.21.15.131:0/3812485072 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f64e0017020 con 0x7f64e4054140 2023-09-12T14:55:04.522 INFO:teuthology.orchestra.run.smithi131.stdout:55834574853 2023-09-12T14:55:04.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.517+0000 7f64d17fa640 1 -- 172.21.15.131:0/3812485072 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f64d8065b20 msgr2=0x7f64d8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.517+0000 7f64d17fa640 1 --2- 172.21.15.131:0/3812485072 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f64d8065b20 0x7f64d8067fe0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f64ec068a70 tx=0x7f64ec08a040 comp rx=0 tx=0).stop 2023-09-12T14:55:04.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.517+0000 7f64d17fa640 1 -- 172.21.15.131:0/3812485072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64e4054140 msgr2=0x7f64e4033cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.517+0000 7f64d17fa640 1 --2- 172.21.15.131:0/3812485072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64e4054140 0x7f64e4033cf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f64e000d980 tx=0x7f64e000de40 comp rx=0 tx=0).stop 2023-09-12T14:55:04.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.517+0000 7f64d17fa640 1 -- 172.21.15.131:0/3812485072 shutdown_connections 2023-09-12T14:55:04.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.517+0000 7f64d17fa640 1 --2- 172.21.15.131:0/3812485072 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f64d8065b20 0x7f64d8067fe0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.517+0000 7f64d17fa640 1 --2- 172.21.15.131:0/3812485072 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64e40df2f0 0x7f64e40e03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.517+0000 7f64d17fa640 1 --2- 172.21.15.131:0/3812485072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64e4054140 0x7f64e4033cf0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.517+0000 7f64d17fa640 1 --2- 172.21.15.131:0/3812485072 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64e4053ce0 0x7f64e4035700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.517+0000 7f64d17fa640 1 -- 172.21.15.131:0/3812485072 >> 172.21.15.131:0/3812485072 conn(0x7f64e4054b10 msgr2=0x7f64e40ed460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:04.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.517+0000 7f64d17fa640 1 -- 172.21.15.131:0/3812485072 shutdown_connections 2023-09-12T14:55:04.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.517+0000 7f64d17fa640 1 -- 172.21.15.131:0/3812485072 wait complete. 2023-09-12T14:55:04.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.533+0000 7f01c8d41640 1 -- 172.21.15.131:0/3568277971 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f01c404e110 con 0x7f01c40b2590 2023-09-12T14:55:04.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.533+0000 7f01abfff640 1 -- 172.21.15.131:0/3568277971 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f01b401d030 con 0x7f01c40b2590 2023-09-12T14:55:04.535 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2023-09-12T14:55:04.536 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2023-09-12T14:55:04.536 INFO:teuthology.orchestra.run.smithi131.stdout:55834574855 2023-09-12T14:55:04.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.537+0000 7f01c8d41640 1 -- 172.21.15.131:0/3568277971 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f01ac065ce0 msgr2=0x7f01ac0681a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.537+0000 7f01c8d41640 1 --2- 172.21.15.131:0/3568277971 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f01ac065ce0 0x7f01ac0681a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00b480 tx=0x7f01bc00d040 comp rx=0 tx=0).stop 2023-09-12T14:55:04.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.537+0000 7f01c8d41640 1 -- 172.21.15.131:0/3568277971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01c40b2590 msgr2=0x7f01c41536a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:04.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.537+0000 7f01c8d41640 1 --2- 172.21.15.131:0/3568277971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01c40b2590 0x7f01c41536a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f01b400c970 tx=0x7f01b400ce30 comp rx=0 tx=0).stop 2023-09-12T14:55:04.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.537+0000 7f01c8d41640 1 -- 172.21.15.131:0/3568277971 shutdown_connections 2023-09-12T14:55:04.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.537+0000 7f01c8d41640 1 --2- 172.21.15.131:0/3568277971 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f01ac065ce0 0x7f01ac0681a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.537+0000 7f01c8d41640 1 --2- 172.21.15.131:0/3568277971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01c41579e0 0x7f01c4166290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.537+0000 7f01c8d41640 1 --2- 172.21.15.131:0/3568277971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01c40b2590 0x7f01c41536a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.537+0000 7f01c8d41640 1 --2- 172.21.15.131:0/3568277971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01c40b15d0 0x7f01c4168320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:04.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.537+0000 7f01c8d41640 1 -- 172.21.15.131:0/3568277971 >> 172.21.15.131:0/3568277971 conn(0x7f01c40cd850 msgr2=0x7f01c4069960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:04.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.537+0000 7f01c8d41640 1 -- 172.21.15.131:0/3568277971 shutdown_connections 2023-09-12T14:55:04.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:04.537+0000 7f01c8d41640 1 -- 172.21.15.131:0/3568277971 wait complete. 2023-09-12T14:55:04.544 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.2 2023-09-12T14:55:04.545 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2023-09-12T14:55:04.548 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2023-09-12T14:55:04.554 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2023-09-12T14:55:04.561 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.4 2023-09-12T14:55:05.515 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-09-12T14:55:05.536 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-09-12T14:55:05.537 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-09-12T14:55:05.545 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-12T14:55:05.545 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-12T14:55:05.549 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-12T14:55:05.555 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-12T14:55:05.561 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-09-12T14:55:05.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.697+0000 7f8a859a5640 1 -- 172.21.15.131:0/909504131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a801677d0 msgr2=0x7f8a800ccf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:05.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.697+0000 7f8a859a5640 1 --2- 172.21.15.131:0/909504131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a801677d0 0x7f8a800ccf20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8a70005fd0 tx=0x7f8a7001d590 comp rx=0 tx=0).stop 2023-09-12T14:55:05.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.701+0000 7f8a859a5640 1 -- 172.21.15.131:0/909504131 shutdown_connections 2023-09-12T14:55:05.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.701+0000 7f8a859a5640 1 --2- 172.21.15.131:0/909504131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a800cd460 0x7f8a801682c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.701+0000 7f8a859a5640 1 --2- 172.21.15.131:0/909504131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a801677d0 0x7f8a800ccf20 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8a70005fd0 tx=0x7f8a7001d590 comp rx=0 tx=0).stop 2023-09-12T14:55:05.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.701+0000 7f8a859a5640 1 -- 172.21.15.131:0/909504131 >> 172.21.15.131:0/909504131 conn(0x7f8a80069640 msgr2=0x7f8a80069a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:05.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.701+0000 7f8a859a5640 1 -- 172.21.15.131:0/909504131 shutdown_connections 2023-09-12T14:55:05.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.701+0000 7f8a859a5640 1 -- 172.21.15.131:0/909504131 wait complete. 2023-09-12T14:55:05.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.701+0000 7f8a859a5640 1 Processor -- start 2023-09-12T14:55:05.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.701+0000 7f8a859a5640 1 -- start start 2023-09-12T14:55:05.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.701+0000 7f8a859a5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a800cd460 0x7f8a8020bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.701+0000 7f8a859a5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a8016c190 0x7f8a80165690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.705+0000 7f8a859a5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a80165bd0 0x7f8a80211d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.705+0000 7f8a7e7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a8016c190 0x7f8a80165690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.705+0000 7f8a7effd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a800cd460 0x7f8a8020bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.705+0000 7f8a7effd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a800cd460 0x7f8a8020bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:42774/0 (socket says 172.21.15.131:42774) 2023-09-12T14:55:05.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.705+0000 7f8a7e7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a8016c190 0x7f8a80165690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39082/0 (socket says 172.21.15.131:39082) 2023-09-12T14:55:05.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.705+0000 7f8a7effd640 1 -- 172.21.15.131:0/2928081501 learned_addr learned my addr 172.21.15.131:0/2928081501 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:05.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.705+0000 7f8a7f7fe640 1 --2- 172.21.15.131:0/2928081501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a80165bd0 0x7f8a80211d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.705+0000 7f8a859a5640 1 -- 172.21.15.131:0/2928081501 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a80166050 con 0x7f8a8016c190 2023-09-12T14:55:05.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.705+0000 7f8a859a5640 1 -- 172.21.15.131:0/2928081501 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a80212aa0 con 0x7f8a80165bd0 2023-09-12T14:55:05.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.705+0000 7f8a859a5640 1 -- 172.21.15.131:0/2928081501 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a80084620 con 0x7f8a800cd460 2023-09-12T14:55:05.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.705+0000 7f8a7e7fc640 1 -- 172.21.15.131:0/2928081501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a800cd460 msgr2=0x7f8a8020bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:05.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.705+0000 7f8a7e7fc640 1 --2- 172.21.15.131:0/2928081501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a800cd460 0x7f8a8020bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.705+0000 7f8a7e7fc640 1 -- 172.21.15.131:0/2928081501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a80165bd0 msgr2=0x7f8a80211d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:05.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.705+0000 7f8a7e7fc640 1 --2- 172.21.15.131:0/2928081501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a80165bd0 0x7f8a80211d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.705+0000 7f8a7e7fc640 1 -- 172.21.15.131:0/2928081501 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a700071c0 con 0x7f8a8016c190 2023-09-12T14:55:05.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.705+0000 7f8a7e7fc640 1 --2- 172.21.15.131:0/2928081501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a8016c190 0x7f8a80165690 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8a7800c6b0 tx=0x7f8a7800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:05.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.705+0000 7f8a67fff640 1 -- 172.21.15.131:0/2928081501 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a78003b80 con 0x7f8a8016c190 2023-09-12T14:55:05.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.705+0000 7f8a859a5640 1 -- 172.21.15.131:0/2928081501 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a80086c80 con 0x7f8a8016c190 2023-09-12T14:55:05.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.705+0000 7f8a859a5640 1 -- 172.21.15.131:0/2928081501 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a80087240 con 0x7f8a8016c190 2023-09-12T14:55:05.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.709+0000 7f8a67fff640 1 -- 172.21.15.131:0/2928081501 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8a78003ce0 con 0x7f8a8016c190 2023-09-12T14:55:05.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.713+0000 7f8a67fff640 1 -- 172.21.15.131:0/2928081501 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a78005330 con 0x7f8a8016c190 2023-09-12T14:55:05.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.713+0000 7f8a67fff640 1 -- 172.21.15.131:0/2928081501 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f8a78005510 con 0x7f8a8016c190 2023-09-12T14:55:05.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.713+0000 7f8a67fff640 1 --2- 172.21.15.131:0/2928081501 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8a6c065ca0 0x7f8a6c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.713+0000 7f8a7effd640 1 --2- 172.21.15.131:0/2928081501 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8a6c065ca0 0x7f8a6c068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.713+0000 7f8a67fff640 1 -- 172.21.15.131:0/2928081501 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a78083270 con 0x7f8a8016c190 2023-09-12T14:55:05.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.713+0000 7f8a859a5640 1 -- 172.21.15.131:0/2928081501 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a60005260 con 0x7f8a8016c190 2023-09-12T14:55:05.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.713+0000 7f8a7effd640 1 --2- 172.21.15.131:0/2928081501 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8a6c065ca0 0x7f8a6c068160 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8a70000c10 tx=0x7f8a70009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:05.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.721+0000 7f8a67fff640 1 -- 172.21.15.131:0/2928081501 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f8a78050c50 con 0x7f8a8016c190 2023-09-12T14:55:05.751 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.749+0000 7f3ae60eb640 1 -- 172.21.15.131:0/828738891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3acc005190 msgr2=0x7f3acc00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:05.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.749+0000 7f3ae60eb640 1 --2- 172.21.15.131:0/828738891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3acc005190 0x7f3acc00b990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8001b00 tx=0x7f3ac801d760 comp rx=0 tx=0).stop 2023-09-12T14:55:05.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.749+0000 7f3ae60eb640 1 -- 172.21.15.131:0/828738891 shutdown_connections 2023-09-12T14:55:05.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.749+0000 7f3ae60eb640 1 --2- 172.21.15.131:0/828738891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3acc005190 0x7f3acc00b990 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.749+0000 7f3ae60eb640 1 -- 172.21.15.131:0/828738891 >> 172.21.15.131:0/828738891 conn(0x7f3ae00ae6b0 msgr2=0x7f3ae00aeac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:05.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.749+0000 7f3ae60eb640 1 -- 172.21.15.131:0/828738891 shutdown_connections 2023-09-12T14:55:05.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.749+0000 7f3ae60eb640 1 -- 172.21.15.131:0/828738891 wait complete. 2023-09-12T14:55:05.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.749+0000 7f3ae60eb640 1 Processor -- start 2023-09-12T14:55:05.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.753+0000 7f3ae60eb640 1 -- start start 2023-09-12T14:55:05.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.753+0000 7f3ae60eb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ae01407d0 0x7f3ae00b1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.753+0000 7f3ae60eb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ae00a65b0 0x7f3ae006c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.753+0000 7f3ae60eb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ae006c820 0x7f3ae00af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.753+0000 7f3ae60eb640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae00a7a30 con 0x7f3ae006c820 2023-09-12T14:55:05.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.753+0000 7f3ae60eb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae006ccd0 con 0x7f3ae01407d0 2023-09-12T14:55:05.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.753+0000 7f3ae60eb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae00a6db0 con 0x7f3ae00a65b0 2023-09-12T14:55:05.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.753+0000 7f3adeffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ae00a65b0 0x7f3ae006c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.753+0000 7f3adeffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ae00a65b0 0x7f3ae006c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:42814/0 (socket says 172.21.15.131:42814) 2023-09-12T14:55:05.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.753+0000 7f3adeffd640 1 -- 172.21.15.131:0/892343214 learned_addr learned my addr 172.21.15.131:0/892343214 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:05.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.753+0000 7f3adf7fe640 1 --2- 172.21.15.131:0/892343214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ae01407d0 0x7f3ae00b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.753+0000 7f3adffff640 1 --2- 172.21.15.131:0/892343214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ae006c820 0x7f3ae00af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.753+0000 7f3adffff640 1 -- 172.21.15.131:0/892343214 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ae00a65b0 msgr2=0x7f3ae006c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:05.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.753+0000 7f3adffff640 1 --2- 172.21.15.131:0/892343214 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ae00a65b0 0x7f3ae006c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.753+0000 7f3adffff640 1 -- 172.21.15.131:0/892343214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ae01407d0 msgr2=0x7f3ae00b1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:05.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.753+0000 7f3adffff640 1 --2- 172.21.15.131:0/892343214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ae01407d0 0x7f3ae00b1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.753+0000 7f3adffff640 1 -- 172.21.15.131:0/892343214 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae00a7090 con 0x7f3ae006c820 2023-09-12T14:55:05.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.753+0000 7f3adffff640 1 --2- 172.21.15.131:0/892343214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ae006c820 0x7f3ae00af7a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3ae015c5e0 tx=0x7f3ac8004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:05.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.753+0000 7f3ad7fff640 1 -- 172.21.15.131:0/892343214 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac80b6070 con 0x7f3ae006c820 2023-09-12T14:55:05.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.757+0000 7f3ad7fff640 1 -- 172.21.15.131:0/892343214 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3ac80aa070 con 0x7f3ae006c820 2023-09-12T14:55:05.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.757+0000 7f3ae60eb640 1 -- 172.21.15.131:0/892343214 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae00a7370 con 0x7f3ae006c820 2023-09-12T14:55:05.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.757+0000 7f3ad7fff640 1 -- 172.21.15.131:0/892343214 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8003860 con 0x7f3ae006c820 2023-09-12T14:55:05.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.757+0000 7f3ae60eb640 1 -- 172.21.15.131:0/892343214 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae00a6a30 con 0x7f3ae006c820 2023-09-12T14:55:05.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.757+0000 7fbc1bfff640 1 -- 172.21.15.131:0/1756699923 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc1c06fc10 msgr2=0x7fbc1c156850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:05.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.757+0000 7fbc1bfff640 1 --2- 172.21.15.131:0/1756699923 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc1c06fc10 0x7fbc1c156850 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbc14009bf0 tx=0x7fbc1401c9f0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.757+0000 7fbc1bfff640 1 -- 172.21.15.131:0/1756699923 shutdown_connections 2023-09-12T14:55:05.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.757+0000 7fbc1bfff640 1 --2- 172.21.15.131:0/1756699923 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc1c06fc10 0x7fbc1c156850 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.757+0000 7fbc1bfff640 1 -- 172.21.15.131:0/1756699923 >> 172.21.15.131:0/1756699923 conn(0x7fbc1c065880 msgr2=0x7fbc1c069240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:05.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.757+0000 7fbc1bfff640 1 -- 172.21.15.131:0/1756699923 shutdown_connections 2023-09-12T14:55:05.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.757+0000 7f3ad7fff640 1 -- 172.21.15.131:0/892343214 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f3ac80ac030 con 0x7f3ae006c820 2023-09-12T14:55:05.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.757+0000 7f8563460640 1 -- 172.21.15.131:0/1321990541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8554051a40 msgr2=0x7f8554051e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:05.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.757+0000 7f8563460640 1 --2- 172.21.15.131:0/1321990541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8554051a40 0x7f8554051e20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f854c009840 tx=0x7f854c01c790 comp rx=0 tx=0).stop 2023-09-12T14:55:05.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.757+0000 7f8563460640 1 -- 172.21.15.131:0/1321990541 shutdown_connections 2023-09-12T14:55:05.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.757+0000 7f8563460640 1 --2- 172.21.15.131:0/1321990541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8554051a40 0x7f8554051e20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.757+0000 7f8563460640 1 -- 172.21.15.131:0/1321990541 >> 172.21.15.131:0/1321990541 conn(0x7f85540537a0 msgr2=0x7f8554053bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:05.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.757+0000 7f8563460640 1 -- 172.21.15.131:0/1321990541 shutdown_connections 2023-09-12T14:55:05.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.757+0000 7f3ad7fff640 1 --2- 172.21.15.131:0/892343214 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3ac0065b50 0x7f3ac0068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.757+0000 7f3adf7fe640 1 --2- 172.21.15.131:0/892343214 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3ac0065b50 0x7f3ac0068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.757+0000 7f3ad7fff640 1 -- 172.21.15.131:0/892343214 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ac811c3a0 con 0x7f3ae006c820 2023-09-12T14:55:05.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.757+0000 7fb38759e640 1 -- 172.21.15.131:0/2263848336 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb38815eab0 msgr2=0x7fb3880cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:05.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.757+0000 7fb38759e640 1 --2- 172.21.15.131:0/2263848336 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb38815eab0 0x7fb3880cc200 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb37c009900 tx=0x7fb37c01c8e0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.757+0000 7fbc1bfff640 1 -- 172.21.15.131:0/1756699923 wait complete. 2023-09-12T14:55:05.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7fbc1bfff640 1 Processor -- start 2023-09-12T14:55:05.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7fb38759e640 1 -- 172.21.15.131:0/2263848336 shutdown_connections 2023-09-12T14:55:05.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7fb38759e640 1 --2- 172.21.15.131:0/2263848336 >> v2:172.21.15.187:3300/0 conn(0x7fb3880cc740 0x7fb38815ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7fb38759e640 1 --2- 172.21.15.131:0/2263848336 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb38815eab0 0x7fb3880cc200 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7fb38759e640 1 -- 172.21.15.131:0/2263848336 >> 172.21.15.131:0/2263848336 conn(0x7fb3880ccb90 msgr2=0x7fb3880ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:05.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f3adf7fe640 1 --2- 172.21.15.131:0/892343214 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3ac0065b50 0x7f3ac0068010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00ca2f0 tx=0x7f3ad80091e0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:05.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.757+0000 7f8563460640 1 -- 172.21.15.131:0/1321990541 wait complete. 2023-09-12T14:55:05.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f8563460640 1 Processor -- start 2023-09-12T14:55:05.776 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f959455c640 1 -- 172.21.15.131:0/1296006363 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95780047d0 msgr2=0x7f9578004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:05.776 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f959455c640 1 --2- 172.21.15.131:0/1296006363 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95780047d0 0x7f9578004c50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f957c006fd0 tx=0x7f957c01cf80 comp rx=0 tx=0).stop 2023-09-12T14:55:05.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f959455c640 1 -- 172.21.15.131:0/1296006363 shutdown_connections 2023-09-12T14:55:05.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7fbc1bfff640 1 -- start start 2023-09-12T14:55:05.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7fbc1bfff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc1c06bc30 0x7fbc1c158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7fbc1bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc1c15eae0 0x7fbc1c169ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f959455c640 1 --2- 172.21.15.131:0/1296006363 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95780047d0 0x7f9578004c50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f8563460640 1 -- start start 2023-09-12T14:55:05.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f959455c640 1 -- 172.21.15.131:0/1296006363 >> 172.21.15.131:0/1296006363 conn(0x7f9584055c20 msgr2=0x7f9584056030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:05.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f959455c640 1 -- 172.21.15.131:0/1296006363 shutdown_connections 2023-09-12T14:55:05.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f959455c640 1 -- 172.21.15.131:0/1296006363 wait complete. 2023-09-12T14:55:05.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7fbc1bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc1c16af40 0x7fbc1c140390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7fbc1a7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc1c15eae0 0x7fbc1c169ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7fbc1a7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc1c15eae0 0x7fbc1c169ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39150/0 (socket says 172.21.15.131:39150) 2023-09-12T14:55:05.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f8563460640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8554035fc0 0x7f85540e0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f8563460640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8554039e50 0x7f855417e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f8563460640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8554051a40 0x7f85540377d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f8563460640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855404e200 con 0x7f8554051a40 2023-09-12T14:55:05.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f8563460640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855417f610 con 0x7f8554039e50 2023-09-12T14:55:05.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f8563460640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f855417f050 con 0x7f8554035fc0 2023-09-12T14:55:05.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7fbc1a7fc640 1 -- 172.21.15.131:0/110786746 learned_addr learned my addr 172.21.15.131:0/110786746 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:05.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7fbc1bfff640 1 -- 172.21.15.131:0/110786746 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c1524b0 con 0x7fbc1c15eae0 2023-09-12T14:55:05.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7fbc1bfff640 1 -- 172.21.15.131:0/110786746 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c152790 con 0x7fbc1c16af40 2023-09-12T14:55:05.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7fbc1bfff640 1 -- 172.21.15.131:0/110786746 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c1696e0 con 0x7fbc1c06bc30 2023-09-12T14:55:05.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f856245e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8554035fc0 0x7f85540e0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f8561c5d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8554039e50 0x7f855417e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f856245e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8554035fc0 0x7f85540e0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:42852/0 (socket says 172.21.15.131:42852) 2023-09-12T14:55:05.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f856245e640 1 -- 172.21.15.131:0/198893010 learned_addr learned my addr 172.21.15.131:0/198893010 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:05.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f8562c5f640 1 --2- 172.21.15.131:0/198893010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8554051a40 0x7f85540377d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7fb38759e640 1 -- 172.21.15.131:0/2263848336 shutdown_connections 2023-09-12T14:55:05.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7fbc1b7fe640 1 --2- 172.21.15.131:0/110786746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc1c16af40 0x7fbc1c140390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7fbc1affd640 1 --2- 172.21.15.131:0/110786746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc1c06bc30 0x7fbc1c158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f8562c5f640 1 -- 172.21.15.131:0/198893010 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8554035fc0 msgr2=0x7f85540e0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:05.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f8562c5f640 1 --2- 172.21.15.131:0/198893010 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8554035fc0 0x7f85540e0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f8562c5f640 1 -- 172.21.15.131:0/198893010 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8554039e50 msgr2=0x7f855417e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:05.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f8562c5f640 1 --2- 172.21.15.131:0/198893010 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8554039e50 0x7f855417e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f8562c5f640 1 -- 172.21.15.131:0/198893010 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f854c0094a0 con 0x7f8554051a40 2023-09-12T14:55:05.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f856245e640 1 --2- 172.21.15.131:0/198893010 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8554035fc0 0x7f85540e0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:55:05.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f8562c5f640 1 --2- 172.21.15.131:0/198893010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8554051a40 0x7f85540377d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f855c0685a0 tx=0x7f855c067430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:05.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7fbc1b7fe640 1 -- 172.21.15.131:0/110786746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc1c06bc30 msgr2=0x7fbc1c158280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:05.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7fbc1b7fe640 1 --2- 172.21.15.131:0/110786746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc1c06bc30 0x7fbc1c158280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7fbc1b7fe640 1 -- 172.21.15.131:0/110786746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc1c15eae0 msgr2=0x7fbc1c169ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:05.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7fbc1b7fe640 1 --2- 172.21.15.131:0/110786746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc1c15eae0 0x7fbc1c169ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7fbc1b7fe640 1 -- 172.21.15.131:0/110786746 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc14009820 con 0x7fbc1c16af40 2023-09-12T14:55:05.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f8552ffd640 1 -- 172.21.15.131:0/198893010 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c06d860 con 0x7f8554051a40 2023-09-12T14:55:05.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f8552ffd640 1 -- 172.21.15.131:0/198893010 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f855c06daf0 con 0x7f8554051a40 2023-09-12T14:55:05.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f8552ffd640 1 -- 172.21.15.131:0/198893010 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c066f50 con 0x7f8554051a40 2023-09-12T14:55:05.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f8563460640 1 -- 172.21.15.131:0/198893010 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855417f330 con 0x7f8554051a40 2023-09-12T14:55:05.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f8563460640 1 -- 172.21.15.131:0/198893010 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85540e1240 con 0x7f8554051a40 2023-09-12T14:55:05.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7fb38759e640 1 -- 172.21.15.131:0/2263848336 wait complete. 2023-09-12T14:55:05.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fb38759e640 1 Processor -- start 2023-09-12T14:55:05.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7f959455c640 1 Processor -- start 2023-09-12T14:55:05.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7f959455c640 1 -- start start 2023-09-12T14:55:05.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7f8552ffd640 1 -- 172.21.15.131:0/198893010 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f855c091480 con 0x7f8554051a40 2023-09-12T14:55:05.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7f8552ffd640 1 --2- 172.21.15.131:0/198893010 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8540065bf0 0x7f85400680b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7f8552ffd640 1 -- 172.21.15.131:0/198893010 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f855c0fb210 con 0x7f8554051a40 2023-09-12T14:55:05.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7f856245e640 1 --2- 172.21.15.131:0/198893010 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8540065bf0 0x7f85400680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7f3715f11640 1 -- 172.21.15.131:0/3090080774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37101413b0 msgr2=0x7f37100cb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:05.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7f3715f11640 1 --2- 172.21.15.131:0/3090080774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37101413b0 0x7f37100cb1f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3708009bc0 tx=0x7f370801ca10 comp rx=0 tx=0).stop 2023-09-12T14:55:05.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fb38759e640 1 -- start start 2023-09-12T14:55:05.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fb38759e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3880cc740 0x7fb3880c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fb38759e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb38815eab0 0x7fb3880af980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fb38759e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb38815f3d0 0x7fb3880b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fb38759e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3880c44c0 con 0x7fb38815eab0 2023-09-12T14:55:05.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fb38759e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3880c4170 con 0x7fb38815f3d0 2023-09-12T14:55:05.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fb38759e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3881f3990 con 0x7fb3880cc740 2023-09-12T14:55:05.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fb386d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb38815f3d0 0x7fb3880b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fb386d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb38815f3d0 0x7fb3880b15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:54084/0 (socket says 172.21.15.131:54084) 2023-09-12T14:55:05.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fb386d9d640 1 -- 172.21.15.131:0/2270220656 learned_addr learned my addr 172.21.15.131:0/2270220656 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:05.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fb38659c640 1 --2- 172.21.15.131:0/2270220656 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3880cc740 0x7fb3880c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fb38659c640 1 -- 172.21.15.131:0/2270220656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb38815f3d0 msgr2=0x7fb3880b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:05.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fb38659c640 1 --2- 172.21.15.131:0/2270220656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb38815f3d0 0x7fb3880b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fb38659c640 1 -- 172.21.15.131:0/2270220656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb38815eab0 msgr2=0x7fb3880af980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:05.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fb38659c640 1 --2- 172.21.15.131:0/2270220656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb38815eab0 0x7fb3880af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fb38659c640 1 -- 172.21.15.131:0/2270220656 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37c009560 con 0x7fb3880cc740 2023-09-12T14:55:05.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fb38659c640 1 --2- 172.21.15.131:0/2270220656 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3880cc740 0x7fb3880c64e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb37c0ac000 tx=0x7fb37c0a5780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:05.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fb376ffd640 1 -- 172.21.15.131:0/2270220656 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c004380 con 0x7fb3880cc740 2023-09-12T14:55:05.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fb376ffd640 1 -- 172.21.15.131:0/2270220656 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb37c005070 con 0x7fb3880cc740 2023-09-12T14:55:05.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fb376ffd640 1 -- 172.21.15.131:0/2270220656 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c005480 con 0x7fb3880cc740 2023-09-12T14:55:05.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fbc1b7fe640 1 --2- 172.21.15.131:0/110786746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc1c16af40 0x7fbc1c140390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbc140ac000 tx=0x7fbc140ae8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:05.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fbc037fe640 1 -- 172.21.15.131:0/110786746 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc140aed80 con 0x7fbc1c16af40 2023-09-12T14:55:05.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fbc037fe640 1 -- 172.21.15.131:0/110786746 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbc140a5870 con 0x7fbc1c16af40 2023-09-12T14:55:05.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fbc037fe640 1 -- 172.21.15.131:0/110786746 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc140a5c60 con 0x7fbc1c16af40 2023-09-12T14:55:05.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fbc1bfff640 1 -- 172.21.15.131:0/110786746 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1c1699c0 con 0x7fbc1c16af40 2023-09-12T14:55:05.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fbc1bfff640 1 -- 172.21.15.131:0/110786746 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1c1587c0 con 0x7fbc1c16af40 2023-09-12T14:55:05.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fbc1a7fc640 1 --2- 172.21.15.131:0/110786746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc1c15eae0 0x7fbc1c169ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:05.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7f18307fe640 1 -- 172.21.15.131:0/3007862299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1828158180 msgr2=0x7f18280cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:05.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7f18307fe640 1 --2- 172.21.15.131:0/3007862299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1828158180 0x7f18280cae70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f182806eea0 tx=0x7f181801c7d0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7f3715f11640 1 -- 172.21.15.131:0/3090080774 shutdown_connections 2023-09-12T14:55:05.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7f3715f11640 1 --2- 172.21.15.131:0/3090080774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f371006fe30 0x7f37100aabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7f3715f11640 1 --2- 172.21.15.131:0/3090080774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37101413b0 0x7f37100cb1f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7f3715f11640 1 -- 172.21.15.131:0/3090080774 >> 172.21.15.131:0/3090080774 conn(0x7f37100ae6b0 msgr2=0x7f37100aeac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:05.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7f3715f11640 1 -- 172.21.15.131:0/3090080774 shutdown_connections 2023-09-12T14:55:05.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fbc1bfff640 1 -- 172.21.15.131:0/110786746 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc1c04e110 con 0x7fbc1c16af40 2023-09-12T14:55:05.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7f3715f11640 1 -- 172.21.15.131:0/3090080774 wait complete. 2023-09-12T14:55:05.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7f959455c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95780047d0 0x7f95840c3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.761+0000 7f3ae60eb640 1 -- 172.21.15.131:0/892343214 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae00a7b70 con 0x7f3ae006c820 2023-09-12T14:55:05.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fb38759e640 1 -- 172.21.15.131:0/2270220656 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3881f3c70 con 0x7fb3880cc740 2023-09-12T14:55:05.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.765+0000 7fb38759e640 1 -- 172.21.15.131:0/2270220656 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3880afec0 con 0x7fb3880cc740 2023-09-12T14:55:05.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f959455c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95840d9010 0x7f9584169a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f18307fe640 1 -- 172.21.15.131:0/3007862299 shutdown_connections 2023-09-12T14:55:05.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f18307fe640 1 --2- 172.21.15.131:0/3007862299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f182806f390 0x7f182806f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f18307fe640 1 --2- 172.21.15.131:0/3007862299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1828158180 0x7f18280cae70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f18307fe640 1 --2- 172.21.15.131:0/3007862299 >> v2:172.21.15.187:3300/0 conn(0x7f18280ab730 0x7f18280abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f18307fe640 1 -- 172.21.15.131:0/3007862299 >> 172.21.15.131:0/3007862299 conn(0x7f1828069840 msgr2=0x7f18280cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:05.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7fb38759e640 1 -- 172.21.15.131:0/2270220656 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3880b0170 con 0x7fb3880cc740 2023-09-12T14:55:05.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f95922f8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95780047d0 0x7f95840c3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f95922f8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95780047d0 0x7f95840c3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:42880/0 (socket says 172.21.15.131:42880) 2023-09-12T14:55:05.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f18307fe640 1 -- 172.21.15.131:0/3007862299 shutdown_connections 2023-09-12T14:55:05.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f95922f8640 1 -- 172.21.15.131:0/3658027046 learned_addr learned my addr 172.21.15.131:0/3658027046 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:05.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f959455c640 1 --2- 172.21.15.131:0/3658027046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95840c5660 0x7f95840c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f9591af7640 1 --2- 172.21.15.131:0/3658027046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95840d9010 0x7f9584169a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f9592af9640 1 --2- 172.21.15.131:0/3658027046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95840c5660 0x7f95840c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f959455c640 1 -- 172.21.15.131:0/3658027046 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95840d9490 con 0x7f95840c5660 2023-09-12T14:55:05.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f959455c640 1 -- 172.21.15.131:0/3658027046 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95840d7410 con 0x7f95840d9010 2023-09-12T14:55:05.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f959455c640 1 -- 172.21.15.131:0/3658027046 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95840c51d0 con 0x7f95780047d0 2023-09-12T14:55:05.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f3715f11640 1 Processor -- start 2023-09-12T14:55:05.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f3715f11640 1 -- start start 2023-09-12T14:55:05.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f18307fe640 1 -- 172.21.15.131:0/3007862299 wait complete. 2023-09-12T14:55:05.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f18307fe640 1 Processor -- start 2023-09-12T14:55:05.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f9592af9640 1 -- 172.21.15.131:0/3658027046 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95780047d0 msgr2=0x7f95840c3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:05.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f9592af9640 1 --2- 172.21.15.131:0/3658027046 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95780047d0 0x7f95840c3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f9592af9640 1 -- 172.21.15.131:0/3658027046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95840d9010 msgr2=0x7f9584169a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:05.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f9592af9640 1 --2- 172.21.15.131:0/3658027046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95840d9010 0x7f9584169a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f9592af9640 1 -- 172.21.15.131:0/3658027046 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95840c45c0 con 0x7f95840c5660 2023-09-12T14:55:05.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f9592af9640 1 --2- 172.21.15.131:0/3658027046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95840c5660 0x7f95840c6030 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f958c068a70 tx=0x7f958c067430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:05.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f8563460640 1 -- 172.21.15.131:0/198893010 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85540057a0 con 0x7f8554051a40 2023-09-12T14:55:05.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f3715f11640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f371006fe30 0x7f37101f1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f3715f11640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37101413b0 0x7f371015a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f3715f11640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f371015c190 0x7f37100aea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f3715f11640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371015ac70 con 0x7f371015c190 2023-09-12T14:55:05.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f3715f11640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37100afca0 con 0x7f37101413b0 2023-09-12T14:55:05.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f3715f11640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37100aff80 con 0x7f371006fe30 2023-09-12T14:55:05.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f370ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f371015c190 0x7f37100aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f370ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f371015c190 0x7f37100aea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39220/0 (socket says 172.21.15.131:39220) 2023-09-12T14:55:05.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f370ffff640 1 -- 172.21.15.131:0/2820324036 learned_addr learned my addr 172.21.15.131:0/2820324036 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:05.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f370f7fe640 1 --2- 172.21.15.131:0/2820324036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f371006fe30 0x7f37101f1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f370effd640 1 --2- 172.21.15.131:0/2820324036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37101413b0 0x7f371015a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f18307fe640 1 -- start start 2023-09-12T14:55:05.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f18307fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f182806f390 0x7f18280a7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f18307fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18280ab730 0x7f18280ac740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f18307fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1828158180 0x7f182814f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f18307fe640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18280a8450 con 0x7f18280ab730 2023-09-12T14:55:05.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f18307fe640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18280a7e90 con 0x7f1828158180 2023-09-12T14:55:05.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f18307fe640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18280a8170 con 0x7f182806f390 2023-09-12T14:55:05.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f182dd99640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18280ab730 0x7f18280ac740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f182dd99640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18280ab730 0x7f18280ac740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39208/0 (socket says 172.21.15.131:39208) 2023-09-12T14:55:05.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f182dd99640 1 -- 172.21.15.131:0/2764611185 learned_addr learned my addr 172.21.15.131:0/2764611185 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:05.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f182ed9b640 1 --2- 172.21.15.131:0/2764611185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1828158180 0x7f182814f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f370ffff640 1 -- 172.21.15.131:0/2820324036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f371006fe30 msgr2=0x7f37101f1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:05.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f370ffff640 1 --2- 172.21.15.131:0/2820324036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f371006fe30 0x7f37101f1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f370ffff640 1 -- 172.21.15.131:0/2820324036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37101413b0 msgr2=0x7f371015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:05.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f370ffff640 1 --2- 172.21.15.131:0/2820324036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37101413b0 0x7f371015a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f856245e640 1 --2- 172.21.15.131:0/198893010 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8540065bf0 0x7f85400680b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f854c009790 tx=0x7f854c01afd0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:05.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f370ffff640 1 -- 172.21.15.131:0/2820324036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3708009820 con 0x7f371015c190 2023-09-12T14:55:05.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7fbc037fe640 1 -- 172.21.15.131:0/110786746 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fbc140b4040 con 0x7fbc1c16af40 2023-09-12T14:55:05.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7fbc037fe640 1 --2- 172.21.15.131:0/110786746 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fbc08065c20 0x7fbc080680e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7fbc037fe640 1 -- 172.21.15.131:0/110786746 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc1411e370 con 0x7fbc1c16af40 2023-09-12T14:55:05.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f182dd99640 1 -- 172.21.15.131:0/2764611185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f182806f390 msgr2=0x7f18280a7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:05.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f182dd99640 1 --2- 172.21.15.131:0/2764611185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f182806f390 0x7f18280a7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f182dd99640 1 -- 172.21.15.131:0/2764611185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1828158180 msgr2=0x7f182814f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:05.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f182dd99640 1 --2- 172.21.15.131:0/2764611185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1828158180 0x7f182814f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:05.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.769+0000 7f182dd99640 1 -- 172.21.15.131:0/2764611185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18180094a0 con 0x7f18280ab730 2023-09-12T14:55:05.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.773+0000 7f182dd99640 1 --2- 172.21.15.131:0/2764611185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18280ab730 0x7f18280ac740 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f181801a400 tx=0x7f18180a5690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:05.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.773+0000 7f370ffff640 1 --2- 172.21.15.131:0/2820324036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f371015c190 0x7f37100aea90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f370000e970 tx=0x7f370000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:05.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.773+0000 7fb376ffd640 1 -- 172.21.15.131:0/2270220656 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fb37c0b4070 con 0x7fb3880cc740 2023-09-12T14:55:05.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.773+0000 7fb376ffd640 1 --2- 172.21.15.131:0/2270220656 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb368065b50 0x7fb368068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.773+0000 7fb376ffd640 1 -- 172.21.15.131:0/2270220656 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb37c11d210 con 0x7fb3880cc740 2023-09-12T14:55:05.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.773+0000 7fb385d9b640 1 --2- 172.21.15.131:0/2270220656 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb368065b50 0x7fb368068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.773+0000 7fb385d9b640 1 --2- 172.21.15.131:0/2270220656 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb368065b50 0x7fb368068010 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb388147e80 tx=0x7fb378006c50 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:05.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.773+0000 7fbc1affd640 1 --2- 172.21.15.131:0/110786746 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fbc08065c20 0x7fbc080680e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.773+0000 7f3ad7fff640 1 -- 172.21.15.131:0/892343214 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f3ae00a7b70 con 0x7f3ae006c820 2023-09-12T14:55:05.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.773+0000 7f9582ffd640 1 -- 172.21.15.131:0/3658027046 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c08d070 con 0x7f95840c5660 2023-09-12T14:55:05.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.773+0000 7f181effd640 1 -- 172.21.15.131:0/2764611185 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1818004230 con 0x7f18280ab730 2023-09-12T14:55:05.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.773+0000 7f36f7fff640 1 -- 172.21.15.131:0/2820324036 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37000096d0 con 0x7f371015c190 2023-09-12T14:55:05.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.773+0000 7f3715f11640 1 -- 172.21.15.131:0/2820324036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37100b0260 con 0x7f371015c190 2023-09-12T14:55:05.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.773+0000 7f3715f11640 1 -- 172.21.15.131:0/2820324036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37100b0820 con 0x7f371015c190 2023-09-12T14:55:05.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.773+0000 7f959455c640 1 -- 172.21.15.131:0/3658027046 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95840c48a0 con 0x7f95840c5660 2023-09-12T14:55:05.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.773+0000 7f959455c640 1 -- 172.21.15.131:0/3658027046 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95840c4e60 con 0x7f95840c5660 2023-09-12T14:55:05.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.773+0000 7f18307fe640 1 -- 172.21.15.131:0/2764611185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18280702b0 con 0x7f18280ab730 2023-09-12T14:55:05.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.773+0000 7f18307fe640 1 -- 172.21.15.131:0/2764611185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f182814f160 con 0x7f18280ab730 2023-09-12T14:55:05.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.777+0000 7f18307fe640 1 -- 172.21.15.131:0/2764611185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1828073580 con 0x7f18280ab730 2023-09-12T14:55:05.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.777+0000 7f36f5ffb640 1 -- 172.21.15.131:0/2820324036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36f0005260 con 0x7f371015c190 2023-09-12T14:55:05.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.777+0000 7fbc1affd640 1 --2- 172.21.15.131:0/110786746 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fbc08065c20 0x7fbc080680e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbc100098e0 tx=0x7fbc10009060 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:05.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.777+0000 7f8552ffd640 1 -- 172.21.15.131:0/198893010 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f855c096070 con 0x7f8554051a40 2023-09-12T14:55:05.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.781+0000 7f9582ffd640 1 -- 172.21.15.131:0/3658027046 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f958c066770 con 0x7f95840c5660 2023-09-12T14:55:05.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.781+0000 7f959455c640 1 -- 172.21.15.131:0/3658027046 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9584006910 con 0x7f95840c5660 2023-09-12T14:55:05.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.781+0000 7f9582ffd640 1 -- 172.21.15.131:0/3658027046 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c06d660 con 0x7f95840c5660 2023-09-12T14:55:05.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.781+0000 7f181effd640 1 -- 172.21.15.131:0/2764611185 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1818004390 con 0x7f18280ab730 2023-09-12T14:55:05.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.781+0000 7f181effd640 1 -- 172.21.15.131:0/2764611185 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18180053a0 con 0x7f18280ab730 2023-09-12T14:55:05.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.781+0000 7f36f7fff640 1 -- 172.21.15.131:0/2820324036 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f370000ccc0 con 0x7f371015c190 2023-09-12T14:55:05.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.781+0000 7fb376ffd640 1 -- 172.21.15.131:0/2270220656 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fb37c0eaad0 con 0x7fb3880cc740 2023-09-12T14:55:05.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.781+0000 7f181effd640 1 -- 172.21.15.131:0/2764611185 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f18180b4040 con 0x7f18280ab730 2023-09-12T14:55:05.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.781+0000 7fbc037fe640 1 -- 172.21.15.131:0/110786746 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fbc140ebde0 con 0x7fbc1c16af40 2023-09-12T14:55:05.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.781+0000 7f181effd640 1 --2- 172.21.15.131:0/2764611185 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f180c065b20 0x7f180c067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.781+0000 7f181effd640 1 -- 172.21.15.131:0/2764611185 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f181811e490 con 0x7f18280ab730 2023-09-12T14:55:05.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.781+0000 7f182e59a640 1 --2- 172.21.15.131:0/2764611185 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f180c065b20 0x7f180c067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.785+0000 7f182e59a640 1 --2- 172.21.15.131:0/2764611185 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f180c065b20 0x7f180c067fe0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f18240058d0 tx=0x7f1824005860 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:05.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.781+0000 7f9582ffd640 1 -- 172.21.15.131:0/3658027046 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f958c06d7c0 con 0x7f95840c5660 2023-09-12T14:55:05.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.785+0000 7f9582ffd640 1 --2- 172.21.15.131:0/3658027046 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9570065ab0 0x7f9570067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.785+0000 7f9582ffd640 1 -- 172.21.15.131:0/3658027046 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f958c0faac0 con 0x7f95840c5660 2023-09-12T14:55:05.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.785+0000 7f95922f8640 1 --2- 172.21.15.131:0/3658027046 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9570065ab0 0x7f9570067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.785+0000 7f95922f8640 1 --2- 172.21.15.131:0/3658027046 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9570065ab0 0x7f9570067f70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9588006840 tx=0x7f9588006b60 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:05.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.785+0000 7f36f7fff640 1 -- 172.21.15.131:0/2820324036 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3700005220 con 0x7f371015c190 2023-09-12T14:55:05.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.785+0000 7f36f7fff640 1 -- 172.21.15.131:0/2820324036 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f3700005440 con 0x7f371015c190 2023-09-12T14:55:05.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.785+0000 7f36f7fff640 1 --2- 172.21.15.131:0/2820324036 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f36fc065b20 0x7f36fc067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:05.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.785+0000 7f36f7fff640 1 -- 172.21.15.131:0/2820324036 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3700014070 con 0x7f371015c190 2023-09-12T14:55:05.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.789+0000 7f370f7fe640 1 --2- 172.21.15.131:0/2820324036 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f36fc065b20 0x7f36fc067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:05.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.789+0000 7f370f7fe640 1 --2- 172.21.15.131:0/2820324036 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f36fc065b20 0x7f36fc067fe0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f37080af000 tx=0x7f370801b250 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:05.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.789+0000 7f9582ffd640 1 -- 172.21.15.131:0/3658027046 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f958c0c8420 con 0x7f95840c5660 2023-09-12T14:55:05.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.797+0000 7f181effd640 1 -- 172.21.15.131:0/2764611185 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f18180ebf10 con 0x7f18280ab730 2023-09-12T14:55:05.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:05.809+0000 7f36f7fff640 1 -- 172.21.15.131:0/2820324036 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f3700051b90 con 0x7f371015c190 2023-09-12T14:55:06.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.021+0000 7f8a859a5640 1 -- 172.21.15.131:0/2928081501 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f8a60005510 con 0x7f8a8016c190 2023-09-12T14:55:06.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.021+0000 7f8a67fff640 1 -- 172.21.15.131:0/2928081501 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8a78001780 con 0x7f8a8016c190 2023-09-12T14:55:06.027 INFO:teuthology.orchestra.run.smithi131.stdout:55834574854 2023-09-12T14:55:06.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.029+0000 7f8a65ffb640 1 -- 172.21.15.131:0/2928081501 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8a6c065ca0 msgr2=0x7f8a6c068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.029+0000 7f8a65ffb640 1 --2- 172.21.15.131:0/2928081501 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8a6c065ca0 0x7f8a6c068160 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8a70000c10 tx=0x7f8a70009040 comp rx=0 tx=0).stop 2023-09-12T14:55:06.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.029+0000 7f8a65ffb640 1 -- 172.21.15.131:0/2928081501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a8016c190 msgr2=0x7f8a80165690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.029+0000 7f8a65ffb640 1 --2- 172.21.15.131:0/2928081501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a8016c190 0x7f8a80165690 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8a7800c6b0 tx=0x7f8a7800cb70 comp rx=0 tx=0).stop 2023-09-12T14:55:06.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.029+0000 7f8a65ffb640 1 -- 172.21.15.131:0/2928081501 shutdown_connections 2023-09-12T14:55:06.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.029+0000 7f8a65ffb640 1 --2- 172.21.15.131:0/2928081501 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8a6c065ca0 0x7f8a6c068160 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.029+0000 7f8a65ffb640 1 --2- 172.21.15.131:0/2928081501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a80165bd0 0x7f8a80211d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.029+0000 7f8a65ffb640 1 --2- 172.21.15.131:0/2928081501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a8016c190 0x7f8a80165690 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.029+0000 7f8a65ffb640 1 --2- 172.21.15.131:0/2928081501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a800cd460 0x7f8a8020bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.029+0000 7f8a65ffb640 1 -- 172.21.15.131:0/2928081501 >> 172.21.15.131:0/2928081501 conn(0x7f8a80069640 msgr2=0x7f8a80171d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:06.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.029+0000 7f8a65ffb640 1 -- 172.21.15.131:0/2928081501 shutdown_connections 2023-09-12T14:55:06.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.029+0000 7f8a65ffb640 1 -- 172.21.15.131:0/2928081501 wait complete. 2023-09-12T14:55:06.053 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2023-09-12T14:55:06.053 DEBUG:teuthology.parallel:result is None 2023-09-12T14:55:06.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.069+0000 7fb38759e640 1 -- 172.21.15.131:0/2270220656 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb3880c8740 con 0x7fb3880cc740 2023-09-12T14:55:06.071 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.069+0000 7f3ae60eb640 1 -- 172.21.15.131:0/892343214 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f3ae00a7b70 con 0x7f3ae006c820 2023-09-12T14:55:06.071 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.069+0000 7f3ad7fff640 1 -- 172.21.15.131:0/892343214 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3ae00a7b70 con 0x7f3ae006c820 2023-09-12T14:55:06.071 INFO:teuthology.orchestra.run.smithi131.stdout:55834574854 2023-09-12T14:55:06.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.069+0000 7fb376ffd640 1 -- 172.21.15.131:0/2270220656 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb3880c8740 con 0x7fb3880cc740 2023-09-12T14:55:06.072 INFO:teuthology.orchestra.run.smithi131.stdout:55834574854 2023-09-12T14:55:06.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.073+0000 7f3ae60eb640 1 -- 172.21.15.131:0/892343214 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3ac0065b50 msgr2=0x7f3ac0068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.073+0000 7f3ae60eb640 1 --2- 172.21.15.131:0/892343214 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3ac0065b50 0x7f3ac0068010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00ca2f0 tx=0x7f3ad80091e0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.073+0000 7f3ae60eb640 1 -- 172.21.15.131:0/892343214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ae006c820 msgr2=0x7f3ae00af7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.073+0000 7fb38759e640 1 -- 172.21.15.131:0/2270220656 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb368065b50 msgr2=0x7fb368068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.073+0000 7fb38759e640 1 --2- 172.21.15.131:0/2270220656 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb368065b50 0x7fb368068010 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb388147e80 tx=0x7fb378006c50 comp rx=0 tx=0).stop 2023-09-12T14:55:06.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.073+0000 7f3ae60eb640 1 --2- 172.21.15.131:0/892343214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ae006c820 0x7f3ae00af7a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3ae015c5e0 tx=0x7f3ac8004f00 comp rx=0 tx=0).stop 2023-09-12T14:55:06.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.073+0000 7fb38759e640 1 -- 172.21.15.131:0/2270220656 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3880cc740 msgr2=0x7fb3880c64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.073+0000 7fb38759e640 1 --2- 172.21.15.131:0/2270220656 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3880cc740 0x7fb3880c64e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb37c0ac000 tx=0x7fb37c0a5780 comp rx=0 tx=0).stop 2023-09-12T14:55:06.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.073+0000 7fb38759e640 1 -- 172.21.15.131:0/2270220656 shutdown_connections 2023-09-12T14:55:06.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.073+0000 7fb38759e640 1 --2- 172.21.15.131:0/2270220656 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb368065b50 0x7fb368068010 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.073+0000 7fb38759e640 1 --2- 172.21.15.131:0/2270220656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb38815f3d0 0x7fb3880b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.073+0000 7fb38759e640 1 --2- 172.21.15.131:0/2270220656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb38815eab0 0x7fb3880af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.073+0000 7fb38759e640 1 --2- 172.21.15.131:0/2270220656 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3880cc740 0x7fb3880c64e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.073+0000 7fb38759e640 1 -- 172.21.15.131:0/2270220656 >> 172.21.15.131:0/2270220656 conn(0x7fb3880ccb90 msgr2=0x7fb38815d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:06.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.073+0000 7fb38759e640 1 -- 172.21.15.131:0/2270220656 shutdown_connections 2023-09-12T14:55:06.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.073+0000 7fb38759e640 1 -- 172.21.15.131:0/2270220656 wait complete. 2023-09-12T14:55:06.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.073+0000 7f3ae60eb640 1 -- 172.21.15.131:0/892343214 shutdown_connections 2023-09-12T14:55:06.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.073+0000 7f3ae60eb640 1 --2- 172.21.15.131:0/892343214 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3ac0065b50 0x7f3ac0068010 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.073+0000 7f3ae60eb640 1 --2- 172.21.15.131:0/892343214 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ae00a65b0 0x7f3ae006c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.073+0000 7f3ae60eb640 1 --2- 172.21.15.131:0/892343214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ae006c820 0x7f3ae00af7a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.073+0000 7f3ae60eb640 1 --2- 172.21.15.131:0/892343214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ae01407d0 0x7f3ae00b1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.073+0000 7f3ae60eb640 1 -- 172.21.15.131:0/892343214 >> 172.21.15.131:0/892343214 conn(0x7f3ae00ae6b0 msgr2=0x7f3ae00acb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:06.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.073+0000 7f3ae60eb640 1 -- 172.21.15.131:0/892343214 shutdown_connections 2023-09-12T14:55:06.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.073+0000 7f3ae60eb640 1 -- 172.21.15.131:0/892343214 wait complete. 2023-09-12T14:55:06.092 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2023-09-12T14:55:06.092 DEBUG:teuthology.parallel:result is None 2023-09-12T14:55:06.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.093+0000 7f8563460640 1 -- 172.21.15.131:0/198893010 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f855404e340 con 0x7f8554051a40 2023-09-12T14:55:06.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.093+0000 7f8552ffd640 1 -- 172.21.15.131:0/198893010 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f855c0c8bf0 con 0x7f8554051a40 2023-09-12T14:55:06.095 INFO:teuthology.orchestra.run.smithi131.stdout:55834574854 2023-09-12T14:55:06.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.093+0000 7f8563460640 1 -- 172.21.15.131:0/198893010 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8540065bf0 msgr2=0x7f85400680b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.093+0000 7f8563460640 1 --2- 172.21.15.131:0/198893010 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8540065bf0 0x7f85400680b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f854c009790 tx=0x7f854c01afd0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.093+0000 7f8563460640 1 -- 172.21.15.131:0/198893010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8554051a40 msgr2=0x7f85540377d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.093+0000 7f8563460640 1 --2- 172.21.15.131:0/198893010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8554051a40 0x7f85540377d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f855c0685a0 tx=0x7f855c067430 comp rx=0 tx=0).stop 2023-09-12T14:55:06.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.093+0000 7f8563460640 1 -- 172.21.15.131:0/198893010 shutdown_connections 2023-09-12T14:55:06.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.093+0000 7f8563460640 1 --2- 172.21.15.131:0/198893010 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8540065bf0 0x7f85400680b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.093+0000 7f8563460640 1 --2- 172.21.15.131:0/198893010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8554051a40 0x7f85540377d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.093+0000 7f8563460640 1 --2- 172.21.15.131:0/198893010 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8554039e50 0x7f855417e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.093+0000 7f8563460640 1 --2- 172.21.15.131:0/198893010 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8554035fc0 0x7f85540e0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.093+0000 7f8563460640 1 -- 172.21.15.131:0/198893010 >> 172.21.15.131:0/198893010 conn(0x7f85540537a0 msgr2=0x7f8554008d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:06.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.093+0000 7f8563460640 1 -- 172.21.15.131:0/198893010 shutdown_connections 2023-09-12T14:55:06.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.093+0000 7f8563460640 1 -- 172.21.15.131:0/198893010 wait complete. 2023-09-12T14:55:06.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.097+0000 7f959455c640 1 -- 172.21.15.131:0/3658027046 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f95840c5360 con 0x7f95840c5660 2023-09-12T14:55:06.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.097+0000 7f9582ffd640 1 -- 172.21.15.131:0/3658027046 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f958c0cba40 con 0x7f95840c5660 2023-09-12T14:55:06.101 INFO:teuthology.orchestra.run.smithi131.stdout:55834574854 2023-09-12T14:55:06.101 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2023-09-12T14:55:06.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.097+0000 7fbc1bfff640 1 -- 172.21.15.131:0/110786746 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbc1c04e110 con 0x7fbc1c16af40 2023-09-12T14:55:06.102 DEBUG:teuthology.parallel:result is None 2023-09-12T14:55:06.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.097+0000 7f959455c640 1 -- 172.21.15.131:0/3658027046 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9570065ab0 msgr2=0x7f9570067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.097+0000 7f959455c640 1 --2- 172.21.15.131:0/3658027046 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9570065ab0 0x7f9570067f70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9588006840 tx=0x7f9588006b60 comp rx=0 tx=0).stop 2023-09-12T14:55:06.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.097+0000 7f959455c640 1 -- 172.21.15.131:0/3658027046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95840c5660 msgr2=0x7f95840c6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.097+0000 7f959455c640 1 --2- 172.21.15.131:0/3658027046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95840c5660 0x7f95840c6030 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f958c068a70 tx=0x7f958c067430 comp rx=0 tx=0).stop 2023-09-12T14:55:06.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.097+0000 7f959455c640 1 -- 172.21.15.131:0/3658027046 shutdown_connections 2023-09-12T14:55:06.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.097+0000 7f959455c640 1 --2- 172.21.15.131:0/3658027046 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9570065ab0 0x7f9570067f70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.097+0000 7f959455c640 1 --2- 172.21.15.131:0/3658027046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95840c5660 0x7f95840c6030 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.097+0000 7f959455c640 1 --2- 172.21.15.131:0/3658027046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95840d9010 0x7f9584169a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.097+0000 7f959455c640 1 --2- 172.21.15.131:0/3658027046 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95780047d0 0x7f95840c3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.101+0000 7f959455c640 1 -- 172.21.15.131:0/3658027046 >> 172.21.15.131:0/3658027046 conn(0x7f9584055c20 msgr2=0x7f95840da120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:06.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.101+0000 7fbc037fe640 1 -- 172.21.15.131:0/110786746 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbc140ba090 con 0x7fbc1c16af40 2023-09-12T14:55:06.105 INFO:teuthology.orchestra.run.smithi131.stdout:55834574854 2023-09-12T14:55:06.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.101+0000 7f959455c640 1 -- 172.21.15.131:0/3658027046 shutdown_connections 2023-09-12T14:55:06.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.101+0000 7f959455c640 1 -- 172.21.15.131:0/3658027046 wait complete. 2023-09-12T14:55:06.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.101+0000 7fbc1bfff640 1 -- 172.21.15.131:0/110786746 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fbc08065c20 msgr2=0x7fbc080680e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.101+0000 7fbc1bfff640 1 --2- 172.21.15.131:0/110786746 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fbc08065c20 0x7fbc080680e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbc100098e0 tx=0x7fbc10009060 comp rx=0 tx=0).stop 2023-09-12T14:55:06.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.101+0000 7fbc1bfff640 1 -- 172.21.15.131:0/110786746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc1c16af40 msgr2=0x7fbc1c140390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.101+0000 7fbc1bfff640 1 --2- 172.21.15.131:0/110786746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc1c16af40 0x7fbc1c140390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbc140ac000 tx=0x7fbc140ae8d0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.101+0000 7fbc1bfff640 1 -- 172.21.15.131:0/110786746 shutdown_connections 2023-09-12T14:55:06.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.101+0000 7fbc1bfff640 1 --2- 172.21.15.131:0/110786746 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fbc08065c20 0x7fbc080680e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.101+0000 7fbc1bfff640 1 --2- 172.21.15.131:0/110786746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc1c16af40 0x7fbc1c140390 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.101+0000 7fbc1bfff640 1 --2- 172.21.15.131:0/110786746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc1c15eae0 0x7fbc1c169ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.101+0000 7fbc1bfff640 1 --2- 172.21.15.131:0/110786746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc1c06bc30 0x7fbc1c158280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.101+0000 7fbc1bfff640 1 -- 172.21.15.131:0/110786746 >> 172.21.15.131:0/110786746 conn(0x7fbc1c065880 msgr2=0x7fbc1c069240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.101+0000 7fbc1bfff640 1 -- 172.21.15.131:0/110786746 shutdown_connections 2023-09-12T14:55:06.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.101+0000 7fbc1bfff640 1 -- 172.21.15.131:0/110786746 wait complete. 2023-09-12T14:55:06.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.105+0000 7f36f5ffb640 1 -- 172.21.15.131:0/2820324036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f36f00057e0 con 0x7f371015c190 2023-09-12T14:55:06.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.109+0000 7f36f7fff640 1 -- 172.21.15.131:0/2820324036 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f37000551b0 con 0x7f371015c190 2023-09-12T14:55:06.111 INFO:teuthology.orchestra.run.smithi131.stdout:55834574853 2023-09-12T14:55:06.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.109+0000 7f36f5ffb640 1 -- 172.21.15.131:0/2820324036 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f36fc065b20 msgr2=0x7f36fc067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.109+0000 7f36f5ffb640 1 --2- 172.21.15.131:0/2820324036 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f36fc065b20 0x7f36fc067fe0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f37080af000 tx=0x7f370801b250 comp rx=0 tx=0).stop 2023-09-12T14:55:06.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.109+0000 7f36f5ffb640 1 -- 172.21.15.131:0/2820324036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f371015c190 msgr2=0x7f37100aea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.109+0000 7f36f5ffb640 1 --2- 172.21.15.131:0/2820324036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f371015c190 0x7f37100aea90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f370000e970 tx=0x7f370000ee30 comp rx=0 tx=0).stop 2023-09-12T14:55:06.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.109+0000 7f36f5ffb640 1 -- 172.21.15.131:0/2820324036 shutdown_connections 2023-09-12T14:55:06.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.109+0000 7f36f5ffb640 1 --2- 172.21.15.131:0/2820324036 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f36fc065b20 0x7f36fc067fe0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.109+0000 7f36f5ffb640 1 --2- 172.21.15.131:0/2820324036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f371015c190 0x7f37100aea90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.109+0000 7f36f5ffb640 1 --2- 172.21.15.131:0/2820324036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37101413b0 0x7f371015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.109+0000 7f36f5ffb640 1 --2- 172.21.15.131:0/2820324036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f371006fe30 0x7f37101f1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.109+0000 7f36f5ffb640 1 -- 172.21.15.131:0/2820324036 >> 172.21.15.131:0/2820324036 conn(0x7f37100ae6b0 msgr2=0x7f37100aca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:06.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.109+0000 7f36f5ffb640 1 -- 172.21.15.131:0/2820324036 shutdown_connections 2023-09-12T14:55:06.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.109+0000 7f36f5ffb640 1 -- 172.21.15.131:0/2820324036 wait complete. 2023-09-12T14:55:06.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.113+0000 7f18307fe640 1 -- 172.21.15.131:0/2764611185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f1828073580 con 0x7f18280ab730 2023-09-12T14:55:06.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.113+0000 7f181effd640 1 -- 172.21.15.131:0/2764611185 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f18180b2020 con 0x7f18280ab730 2023-09-12T14:55:06.116 INFO:teuthology.orchestra.run.smithi131.stdout:55834574856 2023-09-12T14:55:06.118 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.117+0000 7f18307fe640 1 -- 172.21.15.131:0/2764611185 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f180c065b20 msgr2=0x7f180c067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.118 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.117+0000 7f18307fe640 1 --2- 172.21.15.131:0/2764611185 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f180c065b20 0x7f180c067fe0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f18240058d0 tx=0x7f1824005860 comp rx=0 tx=0).stop 2023-09-12T14:55:06.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.117+0000 7f18307fe640 1 -- 172.21.15.131:0/2764611185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18280ab730 msgr2=0x7f18280ac740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.117+0000 7f18307fe640 1 --2- 172.21.15.131:0/2764611185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18280ab730 0x7f18280ac740 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f181801a400 tx=0x7f18180a5690 comp rx=0 tx=0).stop 2023-09-12T14:55:06.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.117+0000 7f18307fe640 1 -- 172.21.15.131:0/2764611185 shutdown_connections 2023-09-12T14:55:06.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.117+0000 7f18307fe640 1 --2- 172.21.15.131:0/2764611185 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f180c065b20 0x7f180c067fe0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.117+0000 7f18307fe640 1 --2- 172.21.15.131:0/2764611185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1828158180 0x7f182814f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.117+0000 7f18307fe640 1 --2- 172.21.15.131:0/2764611185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18280ab730 0x7f18280ac740 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.117+0000 7f18307fe640 1 --2- 172.21.15.131:0/2764611185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f182806f390 0x7f18280a7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.117+0000 7f18307fe640 1 -- 172.21.15.131:0/2764611185 >> 172.21.15.131:0/2764611185 conn(0x7f1828069840 msgr2=0x7f18280aad10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:06.123 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2023-09-12T14:55:06.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.117+0000 7f18307fe640 1 -- 172.21.15.131:0/2764611185 shutdown_connections 2023-09-12T14:55:06.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.117+0000 7f18307fe640 1 -- 172.21.15.131:0/2764611185 wait complete. 2023-09-12T14:55:06.123 DEBUG:teuthology.parallel:result is None 2023-09-12T14:55:06.130 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2023-09-12T14:55:06.130 DEBUG:teuthology.parallel:result is None 2023-09-12T14:55:06.132 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2023-09-12T14:55:06.133 DEBUG:teuthology.parallel:result is None 2023-09-12T14:55:06.139 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2023-09-12T14:55:06.140 DEBUG:teuthology.parallel:result is None 2023-09-12T14:55:06.140 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2023-09-12T14:55:06.141 DEBUG:teuthology.parallel:result is None 2023-09-12T14:55:06.141 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-09-12T14:55:06.141 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-12T14:55:06.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.285+0000 7fef1f986640 1 -- 172.21.15.131:0/196088997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef180abae0 msgr2=0x7fef180c9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.285+0000 7fef1f986640 1 --2- 172.21.15.131:0/196088997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef180abae0 0x7fef180c9310 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fef18071110 tx=0x7fef0801c810 comp rx=0 tx=0).stop 2023-09-12T14:55:06.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.285+0000 7fef1f986640 1 -- 172.21.15.131:0/196088997 shutdown_connections 2023-09-12T14:55:06.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.285+0000 7fef1f986640 1 --2- 172.21.15.131:0/196088997 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef180cc430 0x7fef180c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.285+0000 7fef1f986640 1 --2- 172.21.15.131:0/196088997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef180abae0 0x7fef180c9310 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.285+0000 7fef1f986640 1 --2- 172.21.15.131:0/196088997 >> v2:172.21.15.187:3300/0 conn(0x7fef18158d70 0x7fef1806b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.285+0000 7fef1f986640 1 -- 172.21.15.131:0/196088997 >> 172.21.15.131:0/196088997 conn(0x7fef18069840 msgr2=0x7fef180cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:06.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.285+0000 7fef1f986640 1 -- 172.21.15.131:0/196088997 shutdown_connections 2023-09-12T14:55:06.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.285+0000 7fef1f986640 1 -- 172.21.15.131:0/196088997 wait complete. 2023-09-12T14:55:06.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.289+0000 7fef1f986640 1 Processor -- start 2023-09-12T14:55:06.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.289+0000 7fef1f986640 1 -- start start 2023-09-12T14:55:06.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.289+0000 7fef1f986640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef180abae0 0x7fef18146590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:06.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.289+0000 7fef1f986640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef180cc430 0x7fef1814a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.289+0000 7fef1f986640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef18158d70 0x7fef1814abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.289+0000 7fef1f986640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef18153800 con 0x7fef180abae0 2023-09-12T14:55:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.289+0000 7fef1f986640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef18154810 con 0x7fef18158d70 2023-09-12T14:55:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.289+0000 7fef1f986640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1814b450 con 0x7fef180cc430 2023-09-12T14:55:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.289+0000 7fef1d722640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef180abae0 0x7fef18146590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:06.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.289+0000 7fef1d722640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef180abae0 0x7fef18146590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39238/0 (socket says 172.21.15.131:39238) 2023-09-12T14:55:06.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.289+0000 7fef1d722640 1 -- 172.21.15.131:0/2476217576 learned_addr learned my addr 172.21.15.131:0/2476217576 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:06.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.289+0000 7fef1d722640 1 -- 172.21.15.131:0/2476217576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef180cc430 msgr2=0x7fef1814a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.289+0000 7fef1d722640 1 --2- 172.21.15.131:0/2476217576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef180cc430 0x7fef1814a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.289+0000 7fef1d722640 1 -- 172.21.15.131:0/2476217576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef18158d70 msgr2=0x7fef1814abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.289+0000 7fef1df23640 1 --2- 172.21.15.131:0/2476217576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef18158d70 0x7fef1814abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:06.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.289+0000 7fef1d722640 1 --2- 172.21.15.131:0/2476217576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef18158d70 0x7fef1814abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.289+0000 7fef1d722640 1 -- 172.21.15.131:0/2476217576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef08009560 con 0x7fef180abae0 2023-09-12T14:55:06.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.289+0000 7fef1df23640 1 --2- 172.21.15.131:0/2476217576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef18158d70 0x7fef1814abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:06.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.289+0000 7fef1d722640 1 --2- 172.21.15.131:0/2476217576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef180abae0 0x7fef18146590 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fef1400c9f0 tx=0x7fef1400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:06.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.289+0000 7fef0dffb640 1 -- 172.21.15.131:0/2476217576 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef14007bb0 con 0x7fef180abae0 2023-09-12T14:55:06.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.289+0000 7fef0dffb640 1 -- 172.21.15.131:0/2476217576 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fef14007e00 con 0x7fef180abae0 2023-09-12T14:55:06.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.289+0000 7fef0dffb640 1 -- 172.21.15.131:0/2476217576 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1400f6b0 con 0x7fef180abae0 2023-09-12T14:55:06.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.289+0000 7fef1f986640 1 -- 172.21.15.131:0/2476217576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef18153a50 con 0x7fef180abae0 2023-09-12T14:55:06.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.289+0000 7fef1f986640 1 -- 172.21.15.131:0/2476217576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef18154010 con 0x7fef180abae0 2023-09-12T14:55:06.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.289+0000 7fef1f986640 1 -- 172.21.15.131:0/2476217576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef18073580 con 0x7fef180abae0 2023-09-12T14:55:06.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.289+0000 7fef0dffb640 1 -- 172.21.15.131:0/2476217576 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fef1400f810 con 0x7fef180abae0 2023-09-12T14:55:06.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.293+0000 7fef0dffb640 1 --2- 172.21.15.131:0/2476217576 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7feefc065b20 0x7feefc067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:06.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.293+0000 7fef0dffb640 1 -- 172.21.15.131:0/2476217576 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef14083bb0 con 0x7fef180abae0 2023-09-12T14:55:06.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.293+0000 7fef1cf21640 1 --2- 172.21.15.131:0/2476217576 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7feefc065b20 0x7feefc067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.293+0000 7fef1cf21640 1 --2- 172.21.15.131:0/2476217576 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7feefc065b20 0x7feefc067fe0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fef080ae000 tx=0x7fef080195b0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.297+0000 7fef0dffb640 1 -- 172.21.15.131:0/2476217576 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fef14051480 con 0x7fef180abae0 2023-09-12T14:55:06.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.433+0000 7fef1f986640 1 -- 172.21.15.131:0/2476217576 --> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fef180cd4d0 con 0x7feefc065b20 2023-09-12T14:55:06.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.441+0000 7fef0dffb640 1 -- 172.21.15.131:0/2476217576 <== mgr.4200 v2:172.21.15.131:6832/2298103651 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264534 (secure 0 0 0) 0x7fef180cd4d0 con 0x7feefc065b20 2023-09-12T14:55:06.444 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:06.447 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-09-12T14:55:06.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.441+0000 7fef1f986640 1 -- 172.21.15.131:0/2476217576 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7feefc065b20 msgr2=0x7feefc067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.441+0000 7fef1f986640 1 --2- 172.21.15.131:0/2476217576 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7feefc065b20 0x7feefc067fe0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fef080ae000 tx=0x7fef080195b0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.441+0000 7fef1f986640 1 -- 172.21.15.131:0/2476217576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef180abae0 msgr2=0x7fef18146590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.441+0000 7fef1f986640 1 --2- 172.21.15.131:0/2476217576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef180abae0 0x7fef18146590 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fef1400c9f0 tx=0x7fef1400ceb0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.441+0000 7fef1f986640 1 -- 172.21.15.131:0/2476217576 shutdown_connections 2023-09-12T14:55:06.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.441+0000 7fef1f986640 1 --2- 172.21.15.131:0/2476217576 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7feefc065b20 0x7feefc067fe0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.441+0000 7fef1f986640 1 --2- 172.21.15.131:0/2476217576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef180cc430 0x7fef1814a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.441+0000 7fef1f986640 1 --2- 172.21.15.131:0/2476217576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef18158d70 0x7fef1814abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.441+0000 7fef1f986640 1 --2- 172.21.15.131:0/2476217576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef180abae0 0x7fef18146590 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.441+0000 7fef1f986640 1 -- 172.21.15.131:0/2476217576 >> 172.21.15.131:0/2476217576 conn(0x7fef18069840 msgr2=0x7fef180ad400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:06.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.441+0000 7fef1f986640 1 -- 172.21.15.131:0/2476217576 shutdown_connections 2023-09-12T14:55:06.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.445+0000 7fef1f986640 1 -- 172.21.15.131:0/2476217576 wait complete. 2023-09-12T14:55:06.461 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-12T14:55:05.581367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219012,"kb_used_data":2820,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755708,"statfs":{"total":773094113280,"available":772869844992,"internally_reserved":0,"allocated":2887680,"data_stored":1751721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.918945"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806828+0000","last_change":"2023-09-12T14:54:58.747316+0000","last_active":"2023-09-12T14:54:59.806828+0000","last_peered":"2023-09-12T14:54:59.806828+0000","last_clean":"2023-09-12T14:54:59.806828+0000","last_became_active":"2023-09-12T14:54:58.746402+0000","last_became_peered":"2023-09-12T14:54:58.746402+0000","last_unstale":"2023-09-12T14:54:59.806828+0000","last_undegraded":"2023-09-12T14:54:59.806828+0000","last_fullsized":"2023-09-12T14:54:59.806828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:26:45.799536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.701256+0000","last_change":"2023-09-12T14:54:58.726405+0000","last_active":"2023-09-12T14:54:59.701256+0000","last_peered":"2023-09-12T14:54:59.701256+0000","last_clean":"2023-09-12T14:54:59.701256+0000","last_became_active":"2023-09-12T14:54:58.723598+0000","last_became_peered":"2023-09-12T14:54:58.723598+0000","last_unstale":"2023-09-12T14:54:59.701256+0000","last_undegraded":"2023-09-12T14:54:59.701256+0000","last_fullsized":"2023-09-12T14:54:59.701256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:56:13.426275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806975+0000","last_change":"2023-09-12T14:54:58.735800+0000","last_active":"2023-09-12T14:54:59.806975+0000","last_peered":"2023-09-12T14:54:59.806975+0000","last_clean":"2023-09-12T14:54:59.806975+0000","last_became_active":"2023-09-12T14:54:58.734886+0000","last_became_peered":"2023-09-12T14:54:58.734886+0000","last_unstale":"2023-09-12T14:54:59.806975+0000","last_undegraded":"2023-09-12T14:54:59.806975+0000","last_fullsized":"2023-09-12T14:54:59.806975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:56:13.426275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.705851+0000","last_change":"2023-09-12T14:54:58.751668+0000","last_active":"2023-09-12T14:54:59.705851+0000","last_peered":"2023-09-12T14:54:59.705851+0000","last_clean":"2023-09-12T14:54:59.705851+0000","last_became_active":"2023-09-12T14:54:58.751016+0000","last_became_peered":"2023-09-12T14:54:58.751016+0000","last_unstale":"2023-09-12T14:54:59.705851+0000","last_undegraded":"2023-09-12T14:54:59.705851+0000","last_fullsized":"2023-09-12T14:54:59.705851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:55:04.268808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806474+0000","last_change":"2023-09-12T14:54:58.749393+0000","last_active":"2023-09-12T14:54:59.806474+0000","last_peered":"2023-09-12T14:54:59.806474+0000","last_clean":"2023-09-12T14:54:59.806474+0000","last_became_active":"2023-09-12T14:54:58.748795+0000","last_became_peered":"2023-09-12T14:54:58.748795+0000","last_unstale":"2023-09-12T14:54:59.806474+0000","last_undegraded":"2023-09-12T14:54:59.806474+0000","last_fullsized":"2023-09-12T14:54:59.806474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:18:31.201019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.705919+0000","last_change":"2023-09-12T14:54:58.722071+0000","last_active":"2023-09-12T14:54:59.705919+0000","last_peered":"2023-09-12T14:54:59.705919+0000","last_clean":"2023-09-12T14:54:59.705919+0000","last_became_active":"2023-09-12T14:54:58.721356+0000","last_became_peered":"2023-09-12T14:54:58.721356+0000","last_unstale":"2023-09-12T14:54:59.705919+0000","last_undegraded":"2023-09-12T14:54:59.705919+0000","last_fullsized":"2023-09-12T14:54:59.705919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:26:45.799536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806581+0000","last_change":"2023-09-12T14:54:58.762364+0000","last_active":"2023-09-12T14:54:59.806581+0000","last_peered":"2023-09-12T14:54:59.806581+0000","last_clean":"2023-09-12T14:54:59.806581+0000","last_became_active":"2023-09-12T14:54:58.761809+0000","last_became_peered":"2023-09-12T14:54:58.761809+0000","last_unstale":"2023-09-12T14:54:59.806581+0000","last_undegraded":"2023-09-12T14:54:59.806581+0000","last_fullsized":"2023-09-12T14:54:59.806581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:55:04.268808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807929+0000","last_change":"2023-09-12T14:54:58.749962+0000","last_active":"2023-09-12T14:54:59.807929+0000","last_peered":"2023-09-12T14:54:59.807929+0000","last_clean":"2023-09-12T14:54:59.807929+0000","last_became_active":"2023-09-12T14:54:58.749252+0000","last_became_peered":"2023-09-12T14:54:58.749252+0000","last_unstale":"2023-09-12T14:54:59.807929+0000","last_undegraded":"2023-09-12T14:54:59.807929+0000","last_fullsized":"2023-09-12T14:54:59.807929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:58:08.195627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807390+0000","last_change":"2023-09-12T14:54:58.745069+0000","last_active":"2023-09-12T14:54:59.807390+0000","last_peered":"2023-09-12T14:54:59.807390+0000","last_clean":"2023-09-12T14:54:59.807390+0000","last_became_active":"2023-09-12T14:54:58.744344+0000","last_became_peered":"2023-09-12T14:54:58.744344+0000","last_unstale":"2023-09-12T14:54:59.807390+0000","last_undegraded":"2023-09-12T14:54:59.807390+0000","last_fullsized":"2023-09-12T14:54:59.807390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:07:30.650142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807913+0000","last_change":"2023-09-12T14:54:58.746751+0000","last_active":"2023-09-12T14:54:59.807913+0000","last_peered":"2023-09-12T14:54:59.807913+0000","last_clean":"2023-09-12T14:54:59.807913+0000","last_became_active":"2023-09-12T14:54:58.746118+0000","last_became_peered":"2023-09-12T14:54:58.746118+0000","last_unstale":"2023-09-12T14:54:59.807913+0000","last_undegraded":"2023-09-12T14:54:59.807913+0000","last_fullsized":"2023-09-12T14:54:59.807913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:18:31.201019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807341+0000","last_change":"2023-09-12T14:54:58.744115+0000","last_active":"2023-09-12T14:54:59.807341+0000","last_peered":"2023-09-12T14:54:59.807341+0000","last_clean":"2023-09-12T14:54:59.807341+0000","last_became_active":"2023-09-12T14:54:58.743477+0000","last_became_peered":"2023-09-12T14:54:58.743477+0000","last_unstale":"2023-09-12T14:54:59.807341+0000","last_undegraded":"2023-09-12T14:54:59.807341+0000","last_fullsized":"2023-09-12T14:54:59.807341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:33.647832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.709077+0000","last_change":"2023-09-12T14:54:58.750301+0000","last_active":"2023-09-12T14:54:59.709077+0000","last_peered":"2023-09-12T14:54:59.709077+0000","last_clean":"2023-09-12T14:54:59.709077+0000","last_became_active":"2023-09-12T14:54:58.748251+0000","last_became_peered":"2023-09-12T14:54:58.748251+0000","last_unstale":"2023-09-12T14:54:59.709077+0000","last_undegraded":"2023-09-12T14:54:59.709077+0000","last_fullsized":"2023-09-12T14:54:59.709077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:35.965864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808454+0000","last_change":"2023-09-12T14:54:58.750934+0000","last_active":"2023-09-12T14:54:59.808454+0000","last_peered":"2023-09-12T14:54:59.808454+0000","last_clean":"2023-09-12T14:54:59.808454+0000","last_became_active":"2023-09-12T14:54:58.750202+0000","last_became_peered":"2023-09-12T14:54:58.750202+0000","last_unstale":"2023-09-12T14:54:59.808454+0000","last_undegraded":"2023-09-12T14:54:59.808454+0000","last_fullsized":"2023-09-12T14:54:59.808454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:33.647832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806666+0000","last_change":"2023-09-12T14:54:58.730838+0000","last_active":"2023-09-12T14:54:59.806666+0000","last_peered":"2023-09-12T14:54:59.806666+0000","last_clean":"2023-09-12T14:54:59.806666+0000","last_became_active":"2023-09-12T14:54:58.727571+0000","last_became_peered":"2023-09-12T14:54:58.727571+0000","last_unstale":"2023-09-12T14:54:59.806666+0000","last_undegraded":"2023-09-12T14:54:59.806666+0000","last_fullsized":"2023-09-12T14:54:59.806666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:58:08.195627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.700662+0000","last_change":"2023-09-12T14:54:58.723535+0000","last_active":"2023-09-12T14:54:59.700662+0000","last_peered":"2023-09-12T14:54:59.700662+0000","last_clean":"2023-09-12T14:54:59.700662+0000","last_became_active":"2023-09-12T14:54:58.722775+0000","last_became_peered":"2023-09-12T14:54:58.722775+0000","last_unstale":"2023-09-12T14:54:59.700662+0000","last_undegraded":"2023-09-12T14:54:59.700662+0000","last_fullsized":"2023-09-12T14:54:59.700662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:54:14.557968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.705388+0000","last_change":"2023-09-12T14:54:58.720670+0000","last_active":"2023-09-12T14:54:59.705388+0000","last_peered":"2023-09-12T14:54:59.705388+0000","last_clean":"2023-09-12T14:54:59.705388+0000","last_became_active":"2023-09-12T14:54:58.719787+0000","last_became_peered":"2023-09-12T14:54:58.719787+0000","last_unstale":"2023-09-12T14:54:59.705388+0000","last_undegraded":"2023-09-12T14:54:59.705388+0000","last_fullsized":"2023-09-12T14:54:59.705388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:33:30.961102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.708175+0000","last_change":"2023-09-12T14:54:58.733183+0000","last_active":"2023-09-12T14:54:59.708175+0000","last_peered":"2023-09-12T14:54:59.708175+0000","last_clean":"2023-09-12T14:54:59.708175+0000","last_became_active":"2023-09-12T14:54:58.732383+0000","last_became_peered":"2023-09-12T14:54:58.732383+0000","last_unstale":"2023-09-12T14:54:59.708175+0000","last_undegraded":"2023-09-12T14:54:59.708175+0000","last_fullsized":"2023-09-12T14:54:59.708175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:53:00.606916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805991+0000","last_change":"2023-09-12T14:54:58.751801+0000","last_active":"2023-09-12T14:54:59.805991+0000","last_peered":"2023-09-12T14:54:59.805991+0000","last_clean":"2023-09-12T14:54:59.805991+0000","last_became_active":"2023-09-12T14:54:58.748758+0000","last_became_peered":"2023-09-12T14:54:58.748758+0000","last_unstale":"2023-09-12T14:54:59.805991+0000","last_undegraded":"2023-09-12T14:54:59.805991+0000","last_fullsized":"2023-09-12T14:54:59.805991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:34:39.391000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807256+0000","last_change":"2023-09-12T14:54:58.745955+0000","last_active":"2023-09-12T14:54:59.807256+0000","last_peered":"2023-09-12T14:54:59.807256+0000","last_clean":"2023-09-12T14:54:59.807256+0000","last_became_active":"2023-09-12T14:54:58.744876+0000","last_became_peered":"2023-09-12T14:54:58.744876+0000","last_unstale":"2023-09-12T14:54:59.807256+0000","last_undegraded":"2023-09-12T14:54:59.807256+0000","last_fullsized":"2023-09-12T14:54:59.807256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:13:39.293184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807685+0000","last_change":"2023-09-12T14:54:58.744120+0000","last_active":"2023-09-12T14:54:59.807685+0000","last_peered":"2023-09-12T14:54:59.807685+0000","last_clean":"2023-09-12T14:54:59.807685+0000","last_became_active":"2023-09-12T14:54:58.743538+0000","last_became_peered":"2023-09-12T14:54:58.743538+0000","last_unstale":"2023-09-12T14:54:59.807685+0000","last_undegraded":"2023-09-12T14:54:59.807685+0000","last_fullsized":"2023-09-12T14:54:59.807685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:10:09.339305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806423+0000","last_change":"2023-09-12T14:54:58.733606+0000","last_active":"2023-09-12T14:54:59.806423+0000","last_peered":"2023-09-12T14:54:59.806423+0000","last_clean":"2023-09-12T14:54:59.806423+0000","last_became_active":"2023-09-12T14:54:58.732811+0000","last_became_peered":"2023-09-12T14:54:58.732811+0000","last_unstale":"2023-09-12T14:54:59.806423+0000","last_undegraded":"2023-09-12T14:54:59.806423+0000","last_fullsized":"2023-09-12T14:54:59.806423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:25:02.224632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706033+0000","last_change":"2023-09-12T14:54:58.733719+0000","last_active":"2023-09-12T14:54:59.706033+0000","last_peered":"2023-09-12T14:54:59.706033+0000","last_clean":"2023-09-12T14:54:59.706033+0000","last_became_active":"2023-09-12T14:54:58.733053+0000","last_became_peered":"2023-09-12T14:54:58.733053+0000","last_unstale":"2023-09-12T14:54:59.706033+0000","last_undegraded":"2023-09-12T14:54:59.706033+0000","last_fullsized":"2023-09-12T14:54:59.706033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:16:18.487363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808066+0000","last_change":"2023-09-12T14:54:58.744857+0000","last_active":"2023-09-12T14:54:59.808066+0000","last_peered":"2023-09-12T14:54:59.808066+0000","last_clean":"2023-09-12T14:54:59.808066+0000","last_became_active":"2023-09-12T14:54:58.744250+0000","last_became_peered":"2023-09-12T14:54:58.744250+0000","last_unstale":"2023-09-12T14:54:59.808066+0000","last_undegraded":"2023-09-12T14:54:59.808066+0000","last_fullsized":"2023-09-12T14:54:59.808066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:09:24.488408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.705573+0000","last_change":"2023-09-12T14:54:58.751371+0000","last_active":"2023-09-12T14:54:59.705573+0000","last_peered":"2023-09-12T14:54:59.705573+0000","last_clean":"2023-09-12T14:54:59.705573+0000","last_became_active":"2023-09-12T14:54:58.750585+0000","last_became_peered":"2023-09-12T14:54:58.750585+0000","last_unstale":"2023-09-12T14:54:59.705573+0000","last_undegraded":"2023-09-12T14:54:59.705573+0000","last_fullsized":"2023-09-12T14:54:59.705573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:18:31.201019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.809302+0000","last_change":"2023-09-12T14:54:58.745970+0000","last_active":"2023-09-12T14:54:59.809302+0000","last_peered":"2023-09-12T14:54:59.809302+0000","last_clean":"2023-09-12T14:54:59.809302+0000","last_became_active":"2023-09-12T14:54:58.745163+0000","last_became_peered":"2023-09-12T14:54:58.745163+0000","last_unstale":"2023-09-12T14:54:59.809302+0000","last_undegraded":"2023-09-12T14:54:59.809302+0000","last_fullsized":"2023-09-12T14:54:59.809302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:55:04.268808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807739+0000","last_change":"2023-09-12T14:54:58.746782+0000","last_active":"2023-09-12T14:54:59.807739+0000","last_peered":"2023-09-12T14:54:59.807739+0000","last_clean":"2023-09-12T14:54:59.807739+0000","last_became_active":"2023-09-12T14:54:58.746149+0000","last_became_peered":"2023-09-12T14:54:58.746149+0000","last_unstale":"2023-09-12T14:54:59.807739+0000","last_undegraded":"2023-09-12T14:54:59.807739+0000","last_fullsized":"2023-09-12T14:54:59.807739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:33:30.961102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.701439+0000","last_change":"2023-09-12T14:54:58.747221+0000","last_active":"2023-09-12T14:54:59.701439+0000","last_peered":"2023-09-12T14:54:59.701439+0000","last_clean":"2023-09-12T14:54:59.701439+0000","last_became_active":"2023-09-12T14:54:58.746509+0000","last_became_peered":"2023-09-12T14:54:58.746509+0000","last_unstale":"2023-09-12T14:54:59.701439+0000","last_undegraded":"2023-09-12T14:54:59.701439+0000","last_fullsized":"2023-09-12T14:54:59.701439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:33.647832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808902+0000","last_change":"2023-09-12T14:54:58.754523+0000","last_active":"2023-09-12T14:54:59.808902+0000","last_peered":"2023-09-12T14:54:59.808902+0000","last_clean":"2023-09-12T14:54:59.808902+0000","last_became_active":"2023-09-12T14:54:58.753849+0000","last_became_peered":"2023-09-12T14:54:58.753849+0000","last_unstale":"2023-09-12T14:54:59.808902+0000","last_undegraded":"2023-09-12T14:54:59.808902+0000","last_fullsized":"2023-09-12T14:54:59.808902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:18:31.201019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806568+0000","last_change":"2023-09-12T14:54:58.750572+0000","last_active":"2023-09-12T14:54:59.806568+0000","last_peered":"2023-09-12T14:54:59.806568+0000","last_clean":"2023-09-12T14:54:59.806568+0000","last_became_active":"2023-09-12T14:54:58.750004+0000","last_became_peered":"2023-09-12T14:54:58.750004+0000","last_unstale":"2023-09-12T14:54:59.806568+0000","last_undegraded":"2023-09-12T14:54:59.806568+0000","last_fullsized":"2023-09-12T14:54:59.806568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:54:14.557968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706258+0000","last_change":"2023-09-12T14:54:58.754746+0000","last_active":"2023-09-12T14:54:59.706258+0000","last_peered":"2023-09-12T14:54:59.706258+0000","last_clean":"2023-09-12T14:54:59.706258+0000","last_became_active":"2023-09-12T14:54:58.754081+0000","last_became_peered":"2023-09-12T14:54:58.754081+0000","last_unstale":"2023-09-12T14:54:59.706258+0000","last_undegraded":"2023-09-12T14:54:59.706258+0000","last_fullsized":"2023-09-12T14:54:59.706258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:23:41.038660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805117+0000","last_change":"2023-09-12T14:54:58.739330+0000","last_active":"2023-09-12T14:54:59.805117+0000","last_peered":"2023-09-12T14:54:59.805117+0000","last_clean":"2023-09-12T14:54:59.805117+0000","last_became_active":"2023-09-12T14:54:58.738451+0000","last_became_peered":"2023-09-12T14:54:58.738451+0000","last_unstale":"2023-09-12T14:54:59.805117+0000","last_undegraded":"2023-09-12T14:54:59.805117+0000","last_fullsized":"2023-09-12T14:54:59.805117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:35.965864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805898+0000","last_change":"2023-09-12T14:54:58.745424+0000","last_active":"2023-09-12T14:54:59.805898+0000","last_peered":"2023-09-12T14:54:59.805898+0000","last_clean":"2023-09-12T14:54:59.805898+0000","last_became_active":"2023-09-12T14:54:58.744842+0000","last_became_peered":"2023-09-12T14:54:58.744842+0000","last_unstale":"2023-09-12T14:54:59.805898+0000","last_undegraded":"2023-09-12T14:54:59.805898+0000","last_fullsized":"2023-09-12T14:54:59.805898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:09:24.488408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.708763+0000","last_change":"2023-09-12T14:54:58.734734+0000","last_active":"2023-09-12T14:54:59.708763+0000","last_peered":"2023-09-12T14:54:59.708763+0000","last_clean":"2023-09-12T14:54:59.708763+0000","last_became_active":"2023-09-12T14:54:58.730411+0000","last_became_peered":"2023-09-12T14:54:58.730411+0000","last_unstale":"2023-09-12T14:54:59.708763+0000","last_undegraded":"2023-09-12T14:54:59.708763+0000","last_fullsized":"2023-09-12T14:54:59.708763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:04:39.183616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808747+0000","last_change":"2023-09-12T14:54:57.709617+0000","last_active":"2023-09-12T14:54:59.808747+0000","last_peered":"2023-09-12T14:54:59.808747+0000","last_clean":"2023-09-12T14:54:59.808747+0000","last_became_active":"2023-09-12T14:54:57.709171+0000","last_became_peered":"2023-09-12T14:54:57.709171+0000","last_unstale":"2023-09-12T14:54:59.808747+0000","last_undegraded":"2023-09-12T14:54:59.808747+0000","last_fullsized":"2023-09-12T14:54:59.808747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:35:05.562961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806849+0000","last_change":"2023-09-12T14:54:58.722571+0000","last_active":"2023-09-12T14:54:59.806849+0000","last_peered":"2023-09-12T14:54:59.806849+0000","last_clean":"2023-09-12T14:54:59.806849+0000","last_became_active":"2023-09-12T14:54:58.721782+0000","last_became_peered":"2023-09-12T14:54:58.721782+0000","last_unstale":"2023-09-12T14:54:59.806849+0000","last_undegraded":"2023-09-12T14:54:59.806849+0000","last_fullsized":"2023-09-12T14:54:59.806849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:22:58.799616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807614+0000","last_change":"2023-09-12T14:54:57.709417+0000","last_active":"2023-09-12T14:54:59.807614+0000","last_peered":"2023-09-12T14:54:59.807614+0000","last_clean":"2023-09-12T14:54:59.807614+0000","last_became_active":"2023-09-12T14:54:57.708756+0000","last_became_peered":"2023-09-12T14:54:57.708756+0000","last_unstale":"2023-09-12T14:54:59.807614+0000","last_undegraded":"2023-09-12T14:54:59.807614+0000","last_fullsized":"2023-09-12T14:54:59.807614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:50:58.241369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808240+0000","last_change":"2023-09-12T14:54:57.698971+0000","last_active":"2023-09-12T14:54:59.808240+0000","last_peered":"2023-09-12T14:54:59.808240+0000","last_clean":"2023-09-12T14:54:59.808240+0000","last_became_active":"2023-09-12T14:54:57.698574+0000","last_became_peered":"2023-09-12T14:54:57.698574+0000","last_unstale":"2023-09-12T14:54:59.808240+0000","last_undegraded":"2023-09-12T14:54:59.808240+0000","last_fullsized":"2023-09-12T14:54:59.808240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:25:01.213694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808091+0000","last_change":"2023-09-12T14:54:58.748244+0000","last_active":"2023-09-12T14:54:59.808091+0000","last_peered":"2023-09-12T14:54:59.808091+0000","last_clean":"2023-09-12T14:54:59.808091+0000","last_became_active":"2023-09-12T14:54:58.746823+0000","last_became_peered":"2023-09-12T14:54:58.746823+0000","last_unstale":"2023-09-12T14:54:59.808091+0000","last_undegraded":"2023-09-12T14:54:59.808091+0000","last_fullsized":"2023-09-12T14:54:59.808091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:22:58.799616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706293+0000","last_change":"2023-09-12T14:54:57.708138+0000","last_active":"2023-09-12T14:54:59.706293+0000","last_peered":"2023-09-12T14:54:59.706293+0000","last_clean":"2023-09-12T14:54:59.706293+0000","last_became_active":"2023-09-12T14:54:57.705380+0000","last_became_peered":"2023-09-12T14:54:57.705380+0000","last_unstale":"2023-09-12T14:54:59.706293+0000","last_undegraded":"2023-09-12T14:54:59.706293+0000","last_fullsized":"2023-09-12T14:54:59.706293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:34:38.380062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.700719+0000","last_change":"2023-09-12T14:54:58.751072+0000","last_active":"2023-09-12T14:54:59.700719+0000","last_peered":"2023-09-12T14:54:59.700719+0000","last_clean":"2023-09-12T14:54:59.700719+0000","last_became_active":"2023-09-12T14:54:58.750309+0000","last_became_peered":"2023-09-12T14:54:58.750309+0000","last_unstale":"2023-09-12T14:54:59.700719+0000","last_undegraded":"2023-09-12T14:54:59.700719+0000","last_fullsized":"2023-09-12T14:54:59.700719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:58:08.195627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808058+0000","last_change":"2023-09-12T14:54:57.719821+0000","last_active":"2023-09-12T14:54:59.808058+0000","last_peered":"2023-09-12T14:54:59.808058+0000","last_clean":"2023-09-12T14:54:59.808058+0000","last_became_active":"2023-09-12T14:54:57.717672+0000","last_became_peered":"2023-09-12T14:54:57.717672+0000","last_unstale":"2023-09-12T14:54:59.808058+0000","last_undegraded":"2023-09-12T14:54:59.808058+0000","last_fullsized":"2023-09-12T14:54:59.808058+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:25:01.213694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.707290+0000","last_change":"2023-09-12T14:54:58.745459+0000","last_active":"2023-09-12T14:54:59.707290+0000","last_peered":"2023-09-12T14:54:59.707290+0000","last_clean":"2023-09-12T14:54:59.707290+0000","last_became_active":"2023-09-12T14:54:58.744478+0000","last_became_peered":"2023-09-12T14:54:58.744478+0000","last_unstale":"2023-09-12T14:54:59.707290+0000","last_undegraded":"2023-09-12T14:54:59.707290+0000","last_fullsized":"2023-09-12T14:54:59.707290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:07:30.650142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.707357+0000","last_change":"2023-09-12T14:54:57.712325+0000","last_active":"2023-09-12T14:54:59.707357+0000","last_peered":"2023-09-12T14:54:59.707357+0000","last_clean":"2023-09-12T14:54:59.707357+0000","last_became_active":"2023-09-12T14:54:57.711669+0000","last_became_peered":"2023-09-12T14:54:57.711669+0000","last_unstale":"2023-09-12T14:54:59.707357+0000","last_undegraded":"2023-09-12T14:54:59.707357+0000","last_fullsized":"2023-09-12T14:54:59.707357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:25:01.213694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807090+0000","last_change":"2023-09-12T14:54:58.731110+0000","last_active":"2023-09-12T14:54:59.807090+0000","last_peered":"2023-09-12T14:54:59.807090+0000","last_clean":"2023-09-12T14:54:59.807090+0000","last_became_active":"2023-09-12T14:54:58.728351+0000","last_became_peered":"2023-09-12T14:54:58.728351+0000","last_unstale":"2023-09-12T14:54:59.807090+0000","last_undegraded":"2023-09-12T14:54:59.807090+0000","last_fullsized":"2023-09-12T14:54:59.807090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:10:09.339305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.700987+0000","last_change":"2023-09-12T14:54:57.699372+0000","last_active":"2023-09-12T14:54:59.700987+0000","last_peered":"2023-09-12T14:54:59.700987+0000","last_clean":"2023-09-12T14:54:59.700987+0000","last_became_active":"2023-09-12T14:54:57.698970+0000","last_became_peered":"2023-09-12T14:54:57.698970+0000","last_unstale":"2023-09-12T14:54:59.700987+0000","last_undegraded":"2023-09-12T14:54:59.700987+0000","last_fullsized":"2023-09-12T14:54:59.700987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:50:58.241369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.709034+0000","last_change":"2023-09-12T14:54:58.753601+0000","last_active":"2023-09-12T14:54:59.709034+0000","last_peered":"2023-09-12T14:54:59.709034+0000","last_clean":"2023-09-12T14:54:59.709034+0000","last_became_active":"2023-09-12T14:54:58.752414+0000","last_became_peered":"2023-09-12T14:54:58.752414+0000","last_unstale":"2023-09-12T14:54:59.709034+0000","last_undegraded":"2023-09-12T14:54:59.709034+0000","last_fullsized":"2023-09-12T14:54:59.709034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:41:25.126595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807970+0000","last_change":"2023-09-12T14:54:57.699723+0000","last_active":"2023-09-12T14:54:59.807970+0000","last_peered":"2023-09-12T14:54:59.807970+0000","last_clean":"2023-09-12T14:54:59.807970+0000","last_became_active":"2023-09-12T14:54:57.698942+0000","last_became_peered":"2023-09-12T14:54:57.698942+0000","last_unstale":"2023-09-12T14:54:59.807970+0000","last_undegraded":"2023-09-12T14:54:59.807970+0000","last_fullsized":"2023-09-12T14:54:59.807970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:35:05.562961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806782+0000","last_change":"2023-09-12T14:54:58.756683+0000","last_active":"2023-09-12T14:54:59.806782+0000","last_peered":"2023-09-12T14:54:59.806782+0000","last_clean":"2023-09-12T14:54:59.806782+0000","last_became_active":"2023-09-12T14:54:58.755952+0000","last_became_peered":"2023-09-12T14:54:58.755952+0000","last_unstale":"2023-09-12T14:54:59.806782+0000","last_undegraded":"2023-09-12T14:54:59.806782+0000","last_fullsized":"2023-09-12T14:54:59.806782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:07:26.691746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806819+0000","last_change":"2023-09-12T14:54:57.708009+0000","last_active":"2023-09-12T14:54:59.806819+0000","last_peered":"2023-09-12T14:54:59.806819+0000","last_clean":"2023-09-12T14:54:59.806819+0000","last_became_active":"2023-09-12T14:54:57.707617+0000","last_became_peered":"2023-09-12T14:54:57.707617+0000","last_unstale":"2023-09-12T14:54:59.806819+0000","last_undegraded":"2023-09-12T14:54:59.806819+0000","last_fullsized":"2023-09-12T14:54:59.806819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:10:06.188199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806973+0000","last_change":"2023-09-12T14:54:58.747555+0000","last_active":"2023-09-12T14:54:59.806973+0000","last_peered":"2023-09-12T14:54:59.806973+0000","last_clean":"2023-09-12T14:54:59.806973+0000","last_became_active":"2023-09-12T14:54:58.745580+0000","last_became_peered":"2023-09-12T14:54:58.745580+0000","last_unstale":"2023-09-12T14:54:59.806973+0000","last_undegraded":"2023-09-12T14:54:59.806973+0000","last_fullsized":"2023-09-12T14:54:59.806973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:33.647832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706246+0000","last_change":"2023-09-12T14:54:57.713557+0000","last_active":"2023-09-12T14:54:59.706246+0000","last_peered":"2023-09-12T14:54:59.706246+0000","last_clean":"2023-09-12T14:54:59.706246+0000","last_became_active":"2023-09-12T14:54:57.712038+0000","last_became_peered":"2023-09-12T14:54:57.712038+0000","last_unstale":"2023-09-12T14:54:59.706246+0000","last_undegraded":"2023-09-12T14:54:59.706246+0000","last_fullsized":"2023-09-12T14:54:59.706246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:22:57.788678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806857+0000","last_change":"2023-09-12T14:54:58.754016+0000","last_active":"2023-09-12T14:54:59.806857+0000","last_peered":"2023-09-12T14:54:59.806857+0000","last_clean":"2023-09-12T14:54:59.806857+0000","last_became_active":"2023-09-12T14:54:58.753380+0000","last_became_peered":"2023-09-12T14:54:58.753380+0000","last_unstale":"2023-09-12T14:54:59.806857+0000","last_undegraded":"2023-09-12T14:54:59.806857+0000","last_fullsized":"2023-09-12T14:54:59.806857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:16:18.487363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805823+0000","last_change":"2023-09-12T14:54:57.713543+0000","last_active":"2023-09-12T14:54:59.805823+0000","last_peered":"2023-09-12T14:54:59.805823+0000","last_clean":"2023-09-12T14:54:59.805823+0000","last_became_active":"2023-09-12T14:54:57.712819+0000","last_became_peered":"2023-09-12T14:54:57.712819+0000","last_unstale":"2023-09-12T14:54:59.805823+0000","last_undegraded":"2023-09-12T14:54:59.805823+0000","last_fullsized":"2023-09-12T14:54:59.805823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:10:06.188199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805969+0000","last_change":"2023-09-12T14:54:58.745224+0000","last_active":"2023-09-12T14:54:59.805969+0000","last_peered":"2023-09-12T14:54:59.805969+0000","last_clean":"2023-09-12T14:54:59.805969+0000","last_became_active":"2023-09-12T14:54:58.744566+0000","last_became_peered":"2023-09-12T14:54:58.744566+0000","last_unstale":"2023-09-12T14:54:59.805969+0000","last_undegraded":"2023-09-12T14:54:59.805969+0000","last_fullsized":"2023-09-12T14:54:59.805969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:56:13.426275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805117+0000","last_change":"2023-09-12T14:54:57.683795+0000","last_active":"2023-09-12T14:54:59.805117+0000","last_peered":"2023-09-12T14:54:59.805117+0000","last_clean":"2023-09-12T14:54:59.805117+0000","last_became_active":"2023-09-12T14:54:57.683309+0000","last_became_peered":"2023-09-12T14:54:57.683309+0000","last_unstale":"2023-09-12T14:54:59.805117+0000","last_undegraded":"2023-09-12T14:54:59.805117+0000","last_fullsized":"2023-09-12T14:54:59.805117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:38:46.256403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806420+0000","last_change":"2023-09-12T14:54:58.744197+0000","last_active":"2023-09-12T14:54:59.806420+0000","last_peered":"2023-09-12T14:54:59.806420+0000","last_clean":"2023-09-12T14:54:59.806420+0000","last_became_active":"2023-09-12T14:54:58.743668+0000","last_became_peered":"2023-09-12T14:54:58.743668+0000","last_unstale":"2023-09-12T14:54:59.806420+0000","last_undegraded":"2023-09-12T14:54:59.806420+0000","last_fullsized":"2023-09-12T14:54:59.806420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:33:30.961102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.709354+0000","last_change":"2023-09-12T14:54:57.704349+0000","last_active":"2023-09-12T14:54:59.709354+0000","last_peered":"2023-09-12T14:54:59.709354+0000","last_clean":"2023-09-12T14:54:59.709354+0000","last_became_active":"2023-09-12T14:54:57.703134+0000","last_became_peered":"2023-09-12T14:54:57.703134+0000","last_unstale":"2023-09-12T14:54:59.709354+0000","last_undegraded":"2023-09-12T14:54:59.709354+0000","last_fullsized":"2023-09-12T14:54:59.709354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:25:54.071276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807199+0000","last_change":"2023-09-12T14:54:58.752753+0000","last_active":"2023-09-12T14:54:59.807199+0000","last_peered":"2023-09-12T14:54:59.807199+0000","last_clean":"2023-09-12T14:54:59.807199+0000","last_became_active":"2023-09-12T14:54:58.751852+0000","last_became_peered":"2023-09-12T14:54:58.751852+0000","last_unstale":"2023-09-12T14:54:59.807199+0000","last_undegraded":"2023-09-12T14:54:59.807199+0000","last_fullsized":"2023-09-12T14:54:59.807199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:23:41.038660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806233+0000","last_change":"2023-09-12T14:54:57.691156+0000","last_active":"2023-09-12T14:54:59.806233+0000","last_peered":"2023-09-12T14:54:59.806233+0000","last_clean":"2023-09-12T14:54:59.806233+0000","last_became_active":"2023-09-12T14:54:57.690769+0000","last_became_peered":"2023-09-12T14:54:57.690769+0000","last_unstale":"2023-09-12T14:54:59.806233+0000","last_undegraded":"2023-09-12T14:54:59.806233+0000","last_fullsized":"2023-09-12T14:54:59.806233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:48:28.111452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807364+0000","last_change":"2023-09-12T14:54:58.755613+0000","last_active":"2023-09-12T14:54:59.807364+0000","last_peered":"2023-09-12T14:54:59.807364+0000","last_clean":"2023-09-12T14:54:59.807364+0000","last_became_active":"2023-09-12T14:54:58.755005+0000","last_became_peered":"2023-09-12T14:54:58.755005+0000","last_unstale":"2023-09-12T14:54:59.807364+0000","last_undegraded":"2023-09-12T14:54:59.807364+0000","last_fullsized":"2023-09-12T14:54:59.807364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:27:30.157653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.701127+0000","last_change":"2023-09-12T14:54:57.686468+0000","last_active":"2023-09-12T14:54:59.701127+0000","last_peered":"2023-09-12T14:54:59.701127+0000","last_clean":"2023-09-12T14:54:59.701127+0000","last_became_active":"2023-09-12T14:54:57.686059+0000","last_became_peered":"2023-09-12T14:54:57.686059+0000","last_unstale":"2023-09-12T14:54:59.701127+0000","last_undegraded":"2023-09-12T14:54:59.701127+0000","last_fullsized":"2023-09-12T14:54:59.701127+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:35:05.562961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706190+0000","last_change":"2023-09-12T14:54:58.751002+0000","last_active":"2023-09-12T14:54:59.706190+0000","last_peered":"2023-09-12T14:54:59.706190+0000","last_clean":"2023-09-12T14:54:59.706190+0000","last_became_active":"2023-09-12T14:54:58.750380+0000","last_became_peered":"2023-09-12T14:54:58.750380+0000","last_unstale":"2023-09-12T14:54:59.706190+0000","last_undegraded":"2023-09-12T14:54:59.706190+0000","last_fullsized":"2023-09-12T14:54:59.706190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:56:32.731297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805597+0000","last_change":"2023-09-12T14:54:57.680566+0000","last_active":"2023-09-12T14:54:59.805597+0000","last_peered":"2023-09-12T14:54:59.805597+0000","last_clean":"2023-09-12T14:54:59.805597+0000","last_became_active":"2023-09-12T14:54:57.680113+0000","last_became_peered":"2023-09-12T14:54:57.680113+0000","last_unstale":"2023-09-12T14:54:59.805597+0000","last_undegraded":"2023-09-12T14:54:59.805597+0000","last_fullsized":"2023-09-12T14:54:59.805597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:48:51.271671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806912+0000","last_change":"2023-09-12T14:54:58.746352+0000","last_active":"2023-09-12T14:54:59.806912+0000","last_peered":"2023-09-12T14:54:59.806912+0000","last_clean":"2023-09-12T14:54:59.806912+0000","last_became_active":"2023-09-12T14:54:58.745469+0000","last_became_peered":"2023-09-12T14:54:58.745469+0000","last_unstale":"2023-09-12T14:54:59.806912+0000","last_undegraded":"2023-09-12T14:54:59.806912+0000","last_fullsized":"2023-09-12T14:54:59.806912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:56:32.731297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-12T14:54:58.722614+0000","last_change":"2023-09-12T14:54:46.965446+0000","last_active":"2023-09-12T14:54:58.722614+0000","last_peered":"2023-09-12T14:54:58.722614+0000","last_clean":"2023-09-12T14:54:58.722614+0000","last_became_active":"2023-09-12T14:54:46.963236+0000","last_became_peered":"2023-09-12T14:54:46.963236+0000","last_unstale":"2023-09-12T14:54:58.722614+0000","last_undegraded":"2023-09-12T14:54:58.722614+0000","last_fullsized":"2023-09-12T14:54:58.722614+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:45.804232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:45.804232+0000","last_clean_scrub_stamp":"2023-09-12T14:54:45.804232+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:24:45.670513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.705459+0000","last_change":"2023-09-12T14:54:57.707645+0000","last_active":"2023-09-12T14:54:59.705459+0000","last_peered":"2023-09-12T14:54:59.705459+0000","last_clean":"2023-09-12T14:54:59.705459+0000","last_became_active":"2023-09-12T14:54:57.705665+0000","last_became_peered":"2023-09-12T14:54:57.705665+0000","last_unstale":"2023-09-12T14:54:59.705459+0000","last_undegraded":"2023-09-12T14:54:59.705459+0000","last_fullsized":"2023-09-12T14:54:59.705459+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:48:28.111452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808625+0000","last_change":"2023-09-12T14:54:58.745179+0000","last_active":"2023-09-12T14:54:59.808625+0000","last_peered":"2023-09-12T14:54:59.808625+0000","last_clean":"2023-09-12T14:54:59.808625+0000","last_became_active":"2023-09-12T14:54:58.744548+0000","last_became_peered":"2023-09-12T14:54:58.744548+0000","last_unstale":"2023-09-12T14:54:59.808625+0000","last_undegraded":"2023-09-12T14:54:59.808625+0000","last_fullsized":"2023-09-12T14:54:59.808625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:10:09.339305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806036+0000","last_change":"2023-09-12T14:54:57.719515+0000","last_active":"2023-09-12T14:54:59.806036+0000","last_peered":"2023-09-12T14:54:59.806036+0000","last_clean":"2023-09-12T14:54:59.806036+0000","last_became_active":"2023-09-12T14:54:57.717582+0000","last_became_peered":"2023-09-12T14:54:57.717582+0000","last_unstale":"2023-09-12T14:54:59.806036+0000","last_undegraded":"2023-09-12T14:54:59.806036+0000","last_fullsized":"2023-09-12T14:54:59.806036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:48:51.271671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806471+0000","last_change":"2023-09-12T14:54:58.715137+0000","last_active":"2023-09-12T14:54:59.806471+0000","last_peered":"2023-09-12T14:54:59.806471+0000","last_clean":"2023-09-12T14:54:59.806471+0000","last_became_active":"2023-09-12T14:54:58.713883+0000","last_became_peered":"2023-09-12T14:54:58.713883+0000","last_unstale":"2023-09-12T14:54:59.806471+0000","last_undegraded":"2023-09-12T14:54:59.806471+0000","last_fullsized":"2023-09-12T14:54:59.806471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:20:59.118588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805239+0000","last_change":"2023-09-12T14:54:57.691863+0000","last_active":"2023-09-12T14:54:59.805239+0000","last_peered":"2023-09-12T14:54:59.805239+0000","last_clean":"2023-09-12T14:54:59.805239+0000","last_became_active":"2023-09-12T14:54:57.691453+0000","last_became_peered":"2023-09-12T14:54:57.691453+0000","last_unstale":"2023-09-12T14:54:59.805239+0000","last_undegraded":"2023-09-12T14:54:59.805239+0000","last_fullsized":"2023-09-12T14:54:59.805239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:01:34.927854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.700716+0000","last_change":"2023-09-12T14:54:58.732432+0000","last_active":"2023-09-12T14:54:59.700716+0000","last_peered":"2023-09-12T14:54:59.700716+0000","last_clean":"2023-09-12T14:54:59.700716+0000","last_became_active":"2023-09-12T14:54:58.731662+0000","last_became_peered":"2023-09-12T14:54:58.731662+0000","last_unstale":"2023-09-12T14:54:59.700716+0000","last_undegraded":"2023-09-12T14:54:59.700716+0000","last_fullsized":"2023-09-12T14:54:59.700716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:09:24.488408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706495+0000","last_change":"2023-09-12T14:54:57.714059+0000","last_active":"2023-09-12T14:54:59.706495+0000","last_peered":"2023-09-12T14:54:59.706495+0000","last_clean":"2023-09-12T14:54:59.706495+0000","last_became_active":"2023-09-12T14:54:57.712854+0000","last_became_peered":"2023-09-12T14:54:57.712854+0000","last_unstale":"2023-09-12T14:54:59.706495+0000","last_undegraded":"2023-09-12T14:54:59.706495+0000","last_fullsized":"2023-09-12T14:54:59.706495+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:56:12.415337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.709032+0000","last_change":"2023-09-12T14:54:58.713195+0000","last_active":"2023-09-12T14:54:59.709032+0000","last_peered":"2023-09-12T14:54:59.709032+0000","last_clean":"2023-09-12T14:54:59.709032+0000","last_became_active":"2023-09-12T14:54:58.712392+0000","last_became_peered":"2023-09-12T14:54:58.712392+0000","last_unstale":"2023-09-12T14:54:59.709032+0000","last_undegraded":"2023-09-12T14:54:59.709032+0000","last_fullsized":"2023-09-12T14:54:59.709032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:01:35.938792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.701021+0000","last_change":"2023-09-12T14:54:57.691816+0000","last_active":"2023-09-12T14:54:59.701021+0000","last_peered":"2023-09-12T14:54:59.701021+0000","last_clean":"2023-09-12T14:54:59.701021+0000","last_became_active":"2023-09-12T14:54:57.691410+0000","last_became_peered":"2023-09-12T14:54:57.691410+0000","last_unstale":"2023-09-12T14:54:59.701021+0000","last_undegraded":"2023-09-12T14:54:59.701021+0000","last_fullsized":"2023-09-12T14:54:59.701021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:01:34.927854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706911+0000","last_change":"2023-09-12T14:54:58.748697+0000","last_active":"2023-09-12T14:54:59.706911+0000","last_peered":"2023-09-12T14:54:59.706911+0000","last_clean":"2023-09-12T14:54:59.706911+0000","last_became_active":"2023-09-12T14:54:58.748072+0000","last_became_peered":"2023-09-12T14:54:58.748072+0000","last_unstale":"2023-09-12T14:54:59.706911+0000","last_undegraded":"2023-09-12T14:54:59.706911+0000","last_fullsized":"2023-09-12T14:54:59.706911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:27:30.157653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807241+0000","last_change":"2023-09-12T14:54:57.718970+0000","last_active":"2023-09-12T14:54:59.807241+0000","last_peered":"2023-09-12T14:54:59.807241+0000","last_clean":"2023-09-12T14:54:59.807241+0000","last_became_active":"2023-09-12T14:54:57.717410+0000","last_became_peered":"2023-09-12T14:54:57.717410+0000","last_unstale":"2023-09-12T14:54:59.807241+0000","last_undegraded":"2023-09-12T14:54:59.807241+0000","last_fullsized":"2023-09-12T14:54:59.807241+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:22:57.788678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.701025+0000","last_change":"2023-09-12T14:54:58.733797+0000","last_active":"2023-09-12T14:54:59.701025+0000","last_peered":"2023-09-12T14:54:59.701025+0000","last_clean":"2023-09-12T14:54:59.701025+0000","last_became_active":"2023-09-12T14:54:58.732869+0000","last_became_peered":"2023-09-12T14:54:58.732869+0000","last_unstale":"2023-09-12T14:54:59.701025+0000","last_undegraded":"2023-09-12T14:54:59.701025+0000","last_fullsized":"2023-09-12T14:54:59.701025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:22:58.799616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:55:00.614576+0000","last_change":"2023-09-12T14:54:57.703103+0000","last_active":"2023-09-12T14:55:00.614576+0000","last_peered":"2023-09-12T14:55:00.614576+0000","last_clean":"2023-09-12T14:55:00.614576+0000","last_became_active":"2023-09-12T14:54:57.702706+0000","last_became_peered":"2023-09-12T14:54:57.702706+0000","last_unstale":"2023-09-12T14:55:00.614576+0000","last_undegraded":"2023-09-12T14:55:00.614576+0000","last_fullsized":"2023-09-12T14:55:00.614576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:34:38.380062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.809481+0000","last_change":"2023-09-12T14:54:58.745944+0000","last_active":"2023-09-12T14:54:59.809481+0000","last_peered":"2023-09-12T14:54:59.809481+0000","last_clean":"2023-09-12T14:54:59.809481+0000","last_became_active":"2023-09-12T14:54:58.744677+0000","last_became_peered":"2023-09-12T14:54:58.744677+0000","last_unstale":"2023-09-12T14:54:59.809481+0000","last_undegraded":"2023-09-12T14:54:59.809481+0000","last_fullsized":"2023-09-12T14:54:59.809481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:26:45.799536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808147+0000","last_change":"2023-09-12T14:54:57.719402+0000","last_active":"2023-09-12T14:54:59.808147+0000","last_peered":"2023-09-12T14:54:59.808147+0000","last_clean":"2023-09-12T14:54:59.808147+0000","last_became_active":"2023-09-12T14:54:57.718815+0000","last_became_peered":"2023-09-12T14:54:57.718815+0000","last_unstale":"2023-09-12T14:54:59.808147+0000","last_undegraded":"2023-09-12T14:54:59.808147+0000","last_fullsized":"2023-09-12T14:54:59.808147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:20:58.107650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807282+0000","last_change":"2023-09-12T14:54:58.732619+0000","last_active":"2023-09-12T14:54:59.807282+0000","last_peered":"2023-09-12T14:54:59.807282+0000","last_clean":"2023-09-12T14:54:59.807282+0000","last_became_active":"2023-09-12T14:54:58.731845+0000","last_became_peered":"2023-09-12T14:54:58.731845+0000","last_unstale":"2023-09-12T14:54:59.807282+0000","last_undegraded":"2023-09-12T14:54:59.807282+0000","last_fullsized":"2023-09-12T14:54:59.807282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:33:30.961102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805781+0000","last_change":"2023-09-12T14:54:57.717947+0000","last_active":"2023-09-12T14:54:59.805781+0000","last_peered":"2023-09-12T14:54:59.805781+0000","last_clean":"2023-09-12T14:54:59.805781+0000","last_became_active":"2023-09-12T14:54:57.716465+0000","last_became_peered":"2023-09-12T14:54:57.716465+0000","last_unstale":"2023-09-12T14:54:59.805781+0000","last_undegraded":"2023-09-12T14:54:59.805781+0000","last_fullsized":"2023-09-12T14:54:59.805781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:01:34.927854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.708729+0000","last_change":"2023-09-12T14:54:58.734846+0000","last_active":"2023-09-12T14:54:59.708729+0000","last_peered":"2023-09-12T14:54:59.708729+0000","last_clean":"2023-09-12T14:54:59.708729+0000","last_became_active":"2023-09-12T14:54:58.733232+0000","last_became_peered":"2023-09-12T14:54:58.733232+0000","last_unstale":"2023-09-12T14:54:59.708729+0000","last_undegraded":"2023-09-12T14:54:59.708729+0000","last_fullsized":"2023-09-12T14:54:59.708729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:48:52.282609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.700759+0000","last_change":"2023-09-12T14:54:57.693075+0000","last_active":"2023-09-12T14:54:59.700759+0000","last_peered":"2023-09-12T14:54:59.700759+0000","last_clean":"2023-09-12T14:54:59.700759+0000","last_became_active":"2023-09-12T14:54:57.692190+0000","last_became_peered":"2023-09-12T14:54:57.692190+0000","last_unstale":"2023-09-12T14:54:59.700759+0000","last_undegraded":"2023-09-12T14:54:59.700759+0000","last_fullsized":"2023-09-12T14:54:59.700759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:41:24.115657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808265+0000","last_change":"2023-09-12T14:54:58.748674+0000","last_active":"2023-09-12T14:54:59.808265+0000","last_peered":"2023-09-12T14:54:59.808265+0000","last_clean":"2023-09-12T14:54:59.808265+0000","last_became_active":"2023-09-12T14:54:58.747570+0000","last_became_peered":"2023-09-12T14:54:58.747570+0000","last_unstale":"2023-09-12T14:54:59.808265+0000","last_undegraded":"2023-09-12T14:54:59.808265+0000","last_fullsized":"2023-09-12T14:54:59.808265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:56:13.426275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.700971+0000","last_change":"2023-09-12T14:54:57.690103+0000","last_active":"2023-09-12T14:54:59.700971+0000","last_peered":"2023-09-12T14:54:59.700971+0000","last_clean":"2023-09-12T14:54:59.700971+0000","last_became_active":"2023-09-12T14:54:57.688945+0000","last_became_peered":"2023-09-12T14:54:57.688945+0000","last_unstale":"2023-09-12T14:54:59.700971+0000","last_undegraded":"2023-09-12T14:54:59.700971+0000","last_fullsized":"2023-09-12T14:54:59.700971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:25:01.213694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808786+0000","last_change":"2023-09-12T14:54:58.744905+0000","last_active":"2023-09-12T14:54:59.808786+0000","last_peered":"2023-09-12T14:54:59.808786+0000","last_clean":"2023-09-12T14:54:59.808786+0000","last_became_active":"2023-09-12T14:54:58.744132+0000","last_became_peered":"2023-09-12T14:54:58.744132+0000","last_unstale":"2023-09-12T14:54:59.808786+0000","last_undegraded":"2023-09-12T14:54:59.808786+0000","last_fullsized":"2023-09-12T14:54:59.808786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:33:30.961102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807246+0000","last_change":"2023-09-12T14:54:57.718508+0000","last_active":"2023-09-12T14:54:59.807246+0000","last_peered":"2023-09-12T14:54:59.807246+0000","last_clean":"2023-09-12T14:54:59.807246+0000","last_became_active":"2023-09-12T14:54:57.716751+0000","last_became_peered":"2023-09-12T14:54:57.716751+0000","last_unstale":"2023-09-12T14:54:59.807246+0000","last_undegraded":"2023-09-12T14:54:59.807246+0000","last_fullsized":"2023-09-12T14:54:59.807246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:34:38.380062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805940+0000","last_change":"2023-09-12T14:54:58.747254+0000","last_active":"2023-09-12T14:54:59.805940+0000","last_peered":"2023-09-12T14:54:59.805940+0000","last_clean":"2023-09-12T14:54:59.805940+0000","last_became_active":"2023-09-12T14:54:58.746726+0000","last_became_peered":"2023-09-12T14:54:58.746726+0000","last_unstale":"2023-09-12T14:54:59.805940+0000","last_undegraded":"2023-09-12T14:54:59.805940+0000","last_fullsized":"2023-09-12T14:54:59.805940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:22:58.799616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706879+0000","last_change":"2023-09-12T14:54:58.720140+0000","last_active":"2023-09-12T14:54:59.706879+0000","last_peered":"2023-09-12T14:54:59.706879+0000","last_clean":"2023-09-12T14:54:59.706879+0000","last_became_active":"2023-09-12T14:54:58.717751+0000","last_became_peered":"2023-09-12T14:54:58.717751+0000","last_unstale":"2023-09-12T14:54:59.706879+0000","last_undegraded":"2023-09-12T14:54:59.706879+0000","last_fullsized":"2023-09-12T14:54:59.706879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:13:39.293184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806375+0000","last_change":"2023-09-12T14:54:57.717851+0000","last_active":"2023-09-12T14:54:59.806375+0000","last_peered":"2023-09-12T14:54:59.806375+0000","last_clean":"2023-09-12T14:54:59.806375+0000","last_became_active":"2023-09-12T14:54:57.716821+0000","last_became_peered":"2023-09-12T14:54:57.716821+0000","last_unstale":"2023-09-12T14:54:59.806375+0000","last_undegraded":"2023-09-12T14:54:59.806375+0000","last_fullsized":"2023-09-12T14:54:59.806375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:34.954926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.705907+0000","last_change":"2023-09-12T14:54:58.722319+0000","last_active":"2023-09-12T14:54:59.705907+0000","last_peered":"2023-09-12T14:54:59.705907+0000","last_clean":"2023-09-12T14:54:59.705907+0000","last_became_active":"2023-09-12T14:54:58.719871+0000","last_became_peered":"2023-09-12T14:54:58.719871+0000","last_unstale":"2023-09-12T14:54:59.705907+0000","last_undegraded":"2023-09-12T14:54:59.705907+0000","last_fullsized":"2023-09-12T14:54:59.705907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:07:26.691746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807030+0000","last_change":"2023-09-12T14:54:57.718179+0000","last_active":"2023-09-12T14:54:59.807030+0000","last_peered":"2023-09-12T14:54:59.807030+0000","last_clean":"2023-09-12T14:54:59.807030+0000","last_became_active":"2023-09-12T14:54:57.716712+0000","last_became_peered":"2023-09-12T14:54:57.716712+0000","last_unstale":"2023-09-12T14:54:59.807030+0000","last_undegraded":"2023-09-12T14:54:59.807030+0000","last_fullsized":"2023-09-12T14:54:59.807030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:25:01.213694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806750+0000","last_change":"2023-09-12T14:54:58.728727+0000","last_active":"2023-09-12T14:54:59.806750+0000","last_peered":"2023-09-12T14:54:59.806750+0000","last_clean":"2023-09-12T14:54:59.806750+0000","last_became_active":"2023-09-12T14:54:58.727977+0000","last_became_peered":"2023-09-12T14:54:58.727977+0000","last_unstale":"2023-09-12T14:54:59.806750+0000","last_undegraded":"2023-09-12T14:54:59.806750+0000","last_fullsized":"2023-09-12T14:54:59.806750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:09:24.488408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.809447+0000","last_change":"2023-09-12T14:54:57.683980+0000","last_active":"2023-09-12T14:54:59.809447+0000","last_peered":"2023-09-12T14:54:59.809447+0000","last_clean":"2023-09-12T14:54:59.809447+0000","last_became_active":"2023-09-12T14:54:57.683572+0000","last_became_peered":"2023-09-12T14:54:57.683572+0000","last_unstale":"2023-09-12T14:54:59.809447+0000","last_undegraded":"2023-09-12T14:54:59.809447+0000","last_fullsized":"2023-09-12T14:54:59.809447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:10:06.188199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.809386+0000","last_change":"2023-09-12T14:54:58.758048+0000","last_active":"2023-09-12T14:54:59.809386+0000","last_peered":"2023-09-12T14:54:59.809386+0000","last_clean":"2023-09-12T14:54:59.809386+0000","last_became_active":"2023-09-12T14:54:58.757449+0000","last_became_peered":"2023-09-12T14:54:58.757449+0000","last_unstale":"2023-09-12T14:54:59.809386+0000","last_undegraded":"2023-09-12T14:54:59.809386+0000","last_fullsized":"2023-09-12T14:54:59.809386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:07:26.691746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807331+0000","last_change":"2023-09-12T14:54:57.709447+0000","last_active":"2023-09-12T14:54:59.807331+0000","last_peered":"2023-09-12T14:54:59.807331+0000","last_clean":"2023-09-12T14:54:59.807331+0000","last_became_active":"2023-09-12T14:54:57.709126+0000","last_became_peered":"2023-09-12T14:54:57.709126+0000","last_unstale":"2023-09-12T14:54:59.807331+0000","last_undegraded":"2023-09-12T14:54:59.807331+0000","last_fullsized":"2023-09-12T14:54:59.807331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:34:38.380062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4239999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":70988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":806912,"data_stored":660885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-12T14:55:06.467 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-12T14:55:06.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.621+0000 7f0e98361640 1 -- 172.21.15.131:0/1919014414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e900c98d0 msgr2=0x7f0e900af9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.621+0000 7f0e98361640 1 --2- 172.21.15.131:0/1919014414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e900c98d0 0x7f0e900af9b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0e900b2c70 tx=0x7f0e8801ccf0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.621+0000 7f0e98361640 1 -- 172.21.15.131:0/1919014414 shutdown_connections 2023-09-12T14:55:06.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.621+0000 7f0e98361640 1 --2- 172.21.15.131:0/1919014414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e900c98d0 0x7f0e900af9b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.626 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.621+0000 7f0e98361640 1 --2- 172.21.15.131:0/1919014414 >> v2:172.21.15.187:3300/0 conn(0x7f0e900b2600 0x7f0e900c9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.626 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.621+0000 7f0e98361640 1 -- 172.21.15.131:0/1919014414 >> 172.21.15.131:0/1919014414 conn(0x7f0e90069840 msgr2=0x7f0e900cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:06.626 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.621+0000 7f0e98361640 1 -- 172.21.15.131:0/1919014414 shutdown_connections 2023-09-12T14:55:06.626 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.621+0000 7f0e98361640 1 -- 172.21.15.131:0/1919014414 wait complete. 2023-09-12T14:55:06.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.625+0000 7f0e98361640 1 Processor -- start 2023-09-12T14:55:06.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.625+0000 7f0e98361640 1 -- start start 2023-09-12T14:55:06.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.625+0000 7f0e98361640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e900b2600 0x7f0e90142020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:06.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.625+0000 7f0e98361640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e900c98d0 0x7f0e90147870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:06.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.625+0000 7f0e98361640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e901574d0 0x7f0e90147db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:06.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.625+0000 7f0e98361640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e90142560 con 0x7f0e900b2600 2023-09-12T14:55:06.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.625+0000 7f0e98361640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e90148430 con 0x7f0e901574d0 2023-09-12T14:55:06.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.625+0000 7f0e98361640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e90148710 con 0x7f0e900c98d0 2023-09-12T14:55:06.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.625+0000 7f0e960fd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e900b2600 0x7f0e90142020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:06.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.625+0000 7f0e960fd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e900b2600 0x7f0e90142020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39256/0 (socket says 172.21.15.131:39256) 2023-09-12T14:55:06.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.625+0000 7f0e960fd640 1 -- 172.21.15.131:0/2835895871 learned_addr learned my addr 172.21.15.131:0/2835895871 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:06.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.625+0000 7f0e968fe640 1 --2- 172.21.15.131:0/2835895871 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e901574d0 0x7f0e90147db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:06.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.625+0000 7f0e960fd640 1 -- 172.21.15.131:0/2835895871 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e900c98d0 msgr2=0x7f0e90147870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.625+0000 7f0e958fc640 1 --2- 172.21.15.131:0/2835895871 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e900c98d0 0x7f0e90147870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:06.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.625+0000 7f0e960fd640 1 --2- 172.21.15.131:0/2835895871 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e900c98d0 0x7f0e90147870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.625+0000 7f0e960fd640 1 -- 172.21.15.131:0/2835895871 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e901574d0 msgr2=0x7f0e90147db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.625+0000 7f0e960fd640 1 --2- 172.21.15.131:0/2835895871 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e901574d0 0x7f0e90147db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.625+0000 7f0e960fd640 1 -- 172.21.15.131:0/2835895871 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e880098e0 con 0x7f0e900b2600 2023-09-12T14:55:06.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.625+0000 7f0e968fe640 1 --2- 172.21.15.131:0/2835895871 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e901574d0 0x7f0e90147db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:06.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.625+0000 7f0e960fd640 1 --2- 172.21.15.131:0/2835895871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e900b2600 0x7f0e90142020 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000b410 tx=0x7f0e8000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:06.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.625+0000 7f0e8effd640 1 -- 172.21.15.131:0/2835895871 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e80004020 con 0x7f0e900b2600 2023-09-12T14:55:06.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.625+0000 7f0e8effd640 1 -- 172.21.15.131:0/2835895871 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0e80004270 con 0x7f0e900b2600 2023-09-12T14:55:06.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.625+0000 7f0e98361640 1 -- 172.21.15.131:0/2835895871 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e901489f0 con 0x7f0e900b2600 2023-09-12T14:55:06.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.625+0000 7f0e8effd640 1 -- 172.21.15.131:0/2835895871 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e80011890 con 0x7f0e900b2600 2023-09-12T14:55:06.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.625+0000 7f0e98361640 1 -- 172.21.15.131:0/2835895871 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e901f1cd0 con 0x7f0e900b2600 2023-09-12T14:55:06.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.625+0000 7f0e98361640 1 -- 172.21.15.131:0/2835895871 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e90073580 con 0x7f0e900b2600 2023-09-12T14:55:06.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.625+0000 7f0e8effd640 1 -- 172.21.15.131:0/2835895871 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f0e80003600 con 0x7f0e900b2600 2023-09-12T14:55:06.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.629+0000 7f0e8effd640 1 --2- 172.21.15.131:0/2835895871 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0e7c065b20 0x7f0e7c067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:06.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.629+0000 7f0e8effd640 1 -- 172.21.15.131:0/2835895871 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e800833c0 con 0x7f0e900b2600 2023-09-12T14:55:06.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.629+0000 7f0e958fc640 1 --2- 172.21.15.131:0/2835895871 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0e7c065b20 0x7f0e7c067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:06.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.629+0000 7f0e958fc640 1 --2- 172.21.15.131:0/2835895871 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0e7c065b20 0x7f0e7c067fe0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0e900ae700 tx=0x7f0e84005c70 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:06.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.633+0000 7f0e8effd640 1 -- 172.21.15.131:0/2835895871 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f0e80050c90 con 0x7f0e900b2600 2023-09-12T14:55:06.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.773+0000 7f0e98361640 1 -- 172.21.15.131:0/2835895871 --> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0e90152e50 con 0x7f0e7c065b20 2023-09-12T14:55:06.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.789+0000 7f0e8effd640 1 -- 172.21.15.131:0/2835895871 <== mgr.4200 v2:172.21.15.131:6832/2298103651 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264534 (secure 0 0 0) 0x7f0e90152e50 con 0x7f0e7c065b20 2023-09-12T14:55:06.794 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:06.797 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-09-12T14:55:06.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.793+0000 7f0e98361640 1 -- 172.21.15.131:0/2835895871 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0e7c065b20 msgr2=0x7f0e7c067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.793+0000 7f0e98361640 1 --2- 172.21.15.131:0/2835895871 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0e7c065b20 0x7f0e7c067fe0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0e900ae700 tx=0x7f0e84005c70 comp rx=0 tx=0).stop 2023-09-12T14:55:06.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.793+0000 7f0e98361640 1 -- 172.21.15.131:0/2835895871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e900b2600 msgr2=0x7f0e90142020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.793+0000 7f0e98361640 1 --2- 172.21.15.131:0/2835895871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e900b2600 0x7f0e90142020 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000b410 tx=0x7f0e8000b8d0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.797+0000 7f0e98361640 1 -- 172.21.15.131:0/2835895871 shutdown_connections 2023-09-12T14:55:06.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.797+0000 7f0e98361640 1 --2- 172.21.15.131:0/2835895871 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0e7c065b20 0x7f0e7c067fe0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.797+0000 7f0e98361640 1 --2- 172.21.15.131:0/2835895871 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e900c98d0 0x7f0e90147870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.797+0000 7f0e98361640 1 --2- 172.21.15.131:0/2835895871 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e901574d0 0x7f0e90147db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.797+0000 7f0e98361640 1 --2- 172.21.15.131:0/2835895871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e900b2600 0x7f0e90142020 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.797+0000 7f0e98361640 1 -- 172.21.15.131:0/2835895871 >> 172.21.15.131:0/2835895871 conn(0x7f0e90069840 msgr2=0x7f0e900b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:06.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.797+0000 7f0e98361640 1 -- 172.21.15.131:0/2835895871 shutdown_connections 2023-09-12T14:55:06.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.797+0000 7f0e98361640 1 -- 172.21.15.131:0/2835895871 wait complete. 2023-09-12T14:55:06.813 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-12T14:55:05.581367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219012,"kb_used_data":2820,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755708,"statfs":{"total":773094113280,"available":772869844992,"internally_reserved":0,"allocated":2887680,"data_stored":1751721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.918945"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806828+0000","last_change":"2023-09-12T14:54:58.747316+0000","last_active":"2023-09-12T14:54:59.806828+0000","last_peered":"2023-09-12T14:54:59.806828+0000","last_clean":"2023-09-12T14:54:59.806828+0000","last_became_active":"2023-09-12T14:54:58.746402+0000","last_became_peered":"2023-09-12T14:54:58.746402+0000","last_unstale":"2023-09-12T14:54:59.806828+0000","last_undegraded":"2023-09-12T14:54:59.806828+0000","last_fullsized":"2023-09-12T14:54:59.806828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:26:45.799536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.701256+0000","last_change":"2023-09-12T14:54:58.726405+0000","last_active":"2023-09-12T14:54:59.701256+0000","last_peered":"2023-09-12T14:54:59.701256+0000","last_clean":"2023-09-12T14:54:59.701256+0000","last_became_active":"2023-09-12T14:54:58.723598+0000","last_became_peered":"2023-09-12T14:54:58.723598+0000","last_unstale":"2023-09-12T14:54:59.701256+0000","last_undegraded":"2023-09-12T14:54:59.701256+0000","last_fullsized":"2023-09-12T14:54:59.701256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:56:13.426275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806975+0000","last_change":"2023-09-12T14:54:58.735800+0000","last_active":"2023-09-12T14:54:59.806975+0000","last_peered":"2023-09-12T14:54:59.806975+0000","last_clean":"2023-09-12T14:54:59.806975+0000","last_became_active":"2023-09-12T14:54:58.734886+0000","last_became_peered":"2023-09-12T14:54:58.734886+0000","last_unstale":"2023-09-12T14:54:59.806975+0000","last_undegraded":"2023-09-12T14:54:59.806975+0000","last_fullsized":"2023-09-12T14:54:59.806975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:56:13.426275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.705851+0000","last_change":"2023-09-12T14:54:58.751668+0000","last_active":"2023-09-12T14:54:59.705851+0000","last_peered":"2023-09-12T14:54:59.705851+0000","last_clean":"2023-09-12T14:54:59.705851+0000","last_became_active":"2023-09-12T14:54:58.751016+0000","last_became_peered":"2023-09-12T14:54:58.751016+0000","last_unstale":"2023-09-12T14:54:59.705851+0000","last_undegraded":"2023-09-12T14:54:59.705851+0000","last_fullsized":"2023-09-12T14:54:59.705851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:55:04.268808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806474+0000","last_change":"2023-09-12T14:54:58.749393+0000","last_active":"2023-09-12T14:54:59.806474+0000","last_peered":"2023-09-12T14:54:59.806474+0000","last_clean":"2023-09-12T14:54:59.806474+0000","last_became_active":"2023-09-12T14:54:58.748795+0000","last_became_peered":"2023-09-12T14:54:58.748795+0000","last_unstale":"2023-09-12T14:54:59.806474+0000","last_undegraded":"2023-09-12T14:54:59.806474+0000","last_fullsized":"2023-09-12T14:54:59.806474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:18:31.201019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.705919+0000","last_change":"2023-09-12T14:54:58.722071+0000","last_active":"2023-09-12T14:54:59.705919+0000","last_peered":"2023-09-12T14:54:59.705919+0000","last_clean":"2023-09-12T14:54:59.705919+0000","last_became_active":"2023-09-12T14:54:58.721356+0000","last_became_peered":"2023-09-12T14:54:58.721356+0000","last_unstale":"2023-09-12T14:54:59.705919+0000","last_undegraded":"2023-09-12T14:54:59.705919+0000","last_fullsized":"2023-09-12T14:54:59.705919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:26:45.799536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806581+0000","last_change":"2023-09-12T14:54:58.762364+0000","last_active":"2023-09-12T14:54:59.806581+0000","last_peered":"2023-09-12T14:54:59.806581+0000","last_clean":"2023-09-12T14:54:59.806581+0000","last_became_active":"2023-09-12T14:54:58.761809+0000","last_became_peered":"2023-09-12T14:54:58.761809+0000","last_unstale":"2023-09-12T14:54:59.806581+0000","last_undegraded":"2023-09-12T14:54:59.806581+0000","last_fullsized":"2023-09-12T14:54:59.806581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:55:04.268808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807929+0000","last_change":"2023-09-12T14:54:58.749962+0000","last_active":"2023-09-12T14:54:59.807929+0000","last_peered":"2023-09-12T14:54:59.807929+0000","last_clean":"2023-09-12T14:54:59.807929+0000","last_became_active":"2023-09-12T14:54:58.749252+0000","last_became_peered":"2023-09-12T14:54:58.749252+0000","last_unstale":"2023-09-12T14:54:59.807929+0000","last_undegraded":"2023-09-12T14:54:59.807929+0000","last_fullsized":"2023-09-12T14:54:59.807929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:58:08.195627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807390+0000","last_change":"2023-09-12T14:54:58.745069+0000","last_active":"2023-09-12T14:54:59.807390+0000","last_peered":"2023-09-12T14:54:59.807390+0000","last_clean":"2023-09-12T14:54:59.807390+0000","last_became_active":"2023-09-12T14:54:58.744344+0000","last_became_peered":"2023-09-12T14:54:58.744344+0000","last_unstale":"2023-09-12T14:54:59.807390+0000","last_undegraded":"2023-09-12T14:54:59.807390+0000","last_fullsized":"2023-09-12T14:54:59.807390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:07:30.650142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807913+0000","last_change":"2023-09-12T14:54:58.746751+0000","last_active":"2023-09-12T14:54:59.807913+0000","last_peered":"2023-09-12T14:54:59.807913+0000","last_clean":"2023-09-12T14:54:59.807913+0000","last_became_active":"2023-09-12T14:54:58.746118+0000","last_became_peered":"2023-09-12T14:54:58.746118+0000","last_unstale":"2023-09-12T14:54:59.807913+0000","last_undegraded":"2023-09-12T14:54:59.807913+0000","last_fullsized":"2023-09-12T14:54:59.807913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:18:31.201019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807341+0000","last_change":"2023-09-12T14:54:58.744115+0000","last_active":"2023-09-12T14:54:59.807341+0000","last_peered":"2023-09-12T14:54:59.807341+0000","last_clean":"2023-09-12T14:54:59.807341+0000","last_became_active":"2023-09-12T14:54:58.743477+0000","last_became_peered":"2023-09-12T14:54:58.743477+0000","last_unstale":"2023-09-12T14:54:59.807341+0000","last_undegraded":"2023-09-12T14:54:59.807341+0000","last_fullsized":"2023-09-12T14:54:59.807341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:33.647832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.709077+0000","last_change":"2023-09-12T14:54:58.750301+0000","last_active":"2023-09-12T14:54:59.709077+0000","last_peered":"2023-09-12T14:54:59.709077+0000","last_clean":"2023-09-12T14:54:59.709077+0000","last_became_active":"2023-09-12T14:54:58.748251+0000","last_became_peered":"2023-09-12T14:54:58.748251+0000","last_unstale":"2023-09-12T14:54:59.709077+0000","last_undegraded":"2023-09-12T14:54:59.709077+0000","last_fullsized":"2023-09-12T14:54:59.709077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:35.965864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808454+0000","last_change":"2023-09-12T14:54:58.750934+0000","last_active":"2023-09-12T14:54:59.808454+0000","last_peered":"2023-09-12T14:54:59.808454+0000","last_clean":"2023-09-12T14:54:59.808454+0000","last_became_active":"2023-09-12T14:54:58.750202+0000","last_became_peered":"2023-09-12T14:54:58.750202+0000","last_unstale":"2023-09-12T14:54:59.808454+0000","last_undegraded":"2023-09-12T14:54:59.808454+0000","last_fullsized":"2023-09-12T14:54:59.808454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:33.647832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806666+0000","last_change":"2023-09-12T14:54:58.730838+0000","last_active":"2023-09-12T14:54:59.806666+0000","last_peered":"2023-09-12T14:54:59.806666+0000","last_clean":"2023-09-12T14:54:59.806666+0000","last_became_active":"2023-09-12T14:54:58.727571+0000","last_became_peered":"2023-09-12T14:54:58.727571+0000","last_unstale":"2023-09-12T14:54:59.806666+0000","last_undegraded":"2023-09-12T14:54:59.806666+0000","last_fullsized":"2023-09-12T14:54:59.806666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:58:08.195627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.700662+0000","last_change":"2023-09-12T14:54:58.723535+0000","last_active":"2023-09-12T14:54:59.700662+0000","last_peered":"2023-09-12T14:54:59.700662+0000","last_clean":"2023-09-12T14:54:59.700662+0000","last_became_active":"2023-09-12T14:54:58.722775+0000","last_became_peered":"2023-09-12T14:54:58.722775+0000","last_unstale":"2023-09-12T14:54:59.700662+0000","last_undegraded":"2023-09-12T14:54:59.700662+0000","last_fullsized":"2023-09-12T14:54:59.700662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:54:14.557968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.705388+0000","last_change":"2023-09-12T14:54:58.720670+0000","last_active":"2023-09-12T14:54:59.705388+0000","last_peered":"2023-09-12T14:54:59.705388+0000","last_clean":"2023-09-12T14:54:59.705388+0000","last_became_active":"2023-09-12T14:54:58.719787+0000","last_became_peered":"2023-09-12T14:54:58.719787+0000","last_unstale":"2023-09-12T14:54:59.705388+0000","last_undegraded":"2023-09-12T14:54:59.705388+0000","last_fullsized":"2023-09-12T14:54:59.705388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:33:30.961102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.708175+0000","last_change":"2023-09-12T14:54:58.733183+0000","last_active":"2023-09-12T14:54:59.708175+0000","last_peered":"2023-09-12T14:54:59.708175+0000","last_clean":"2023-09-12T14:54:59.708175+0000","last_became_active":"2023-09-12T14:54:58.732383+0000","last_became_peered":"2023-09-12T14:54:58.732383+0000","last_unstale":"2023-09-12T14:54:59.708175+0000","last_undegraded":"2023-09-12T14:54:59.708175+0000","last_fullsized":"2023-09-12T14:54:59.708175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:53:00.606916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805991+0000","last_change":"2023-09-12T14:54:58.751801+0000","last_active":"2023-09-12T14:54:59.805991+0000","last_peered":"2023-09-12T14:54:59.805991+0000","last_clean":"2023-09-12T14:54:59.805991+0000","last_became_active":"2023-09-12T14:54:58.748758+0000","last_became_peered":"2023-09-12T14:54:58.748758+0000","last_unstale":"2023-09-12T14:54:59.805991+0000","last_undegraded":"2023-09-12T14:54:59.805991+0000","last_fullsized":"2023-09-12T14:54:59.805991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:34:39.391000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807256+0000","last_change":"2023-09-12T14:54:58.745955+0000","last_active":"2023-09-12T14:54:59.807256+0000","last_peered":"2023-09-12T14:54:59.807256+0000","last_clean":"2023-09-12T14:54:59.807256+0000","last_became_active":"2023-09-12T14:54:58.744876+0000","last_became_peered":"2023-09-12T14:54:58.744876+0000","last_unstale":"2023-09-12T14:54:59.807256+0000","last_undegraded":"2023-09-12T14:54:59.807256+0000","last_fullsized":"2023-09-12T14:54:59.807256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:13:39.293184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807685+0000","last_change":"2023-09-12T14:54:58.744120+0000","last_active":"2023-09-12T14:54:59.807685+0000","last_peered":"2023-09-12T14:54:59.807685+0000","last_clean":"2023-09-12T14:54:59.807685+0000","last_became_active":"2023-09-12T14:54:58.743538+0000","last_became_peered":"2023-09-12T14:54:58.743538+0000","last_unstale":"2023-09-12T14:54:59.807685+0000","last_undegraded":"2023-09-12T14:54:59.807685+0000","last_fullsized":"2023-09-12T14:54:59.807685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:10:09.339305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806423+0000","last_change":"2023-09-12T14:54:58.733606+0000","last_active":"2023-09-12T14:54:59.806423+0000","last_peered":"2023-09-12T14:54:59.806423+0000","last_clean":"2023-09-12T14:54:59.806423+0000","last_became_active":"2023-09-12T14:54:58.732811+0000","last_became_peered":"2023-09-12T14:54:58.732811+0000","last_unstale":"2023-09-12T14:54:59.806423+0000","last_undegraded":"2023-09-12T14:54:59.806423+0000","last_fullsized":"2023-09-12T14:54:59.806423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:25:02.224632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706033+0000","last_change":"2023-09-12T14:54:58.733719+0000","last_active":"2023-09-12T14:54:59.706033+0000","last_peered":"2023-09-12T14:54:59.706033+0000","last_clean":"2023-09-12T14:54:59.706033+0000","last_became_active":"2023-09-12T14:54:58.733053+0000","last_became_peered":"2023-09-12T14:54:58.733053+0000","last_unstale":"2023-09-12T14:54:59.706033+0000","last_undegraded":"2023-09-12T14:54:59.706033+0000","last_fullsized":"2023-09-12T14:54:59.706033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:16:18.487363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808066+0000","last_change":"2023-09-12T14:54:58.744857+0000","last_active":"2023-09-12T14:54:59.808066+0000","last_peered":"2023-09-12T14:54:59.808066+0000","last_clean":"2023-09-12T14:54:59.808066+0000","last_became_active":"2023-09-12T14:54:58.744250+0000","last_became_peered":"2023-09-12T14:54:58.744250+0000","last_unstale":"2023-09-12T14:54:59.808066+0000","last_undegraded":"2023-09-12T14:54:59.808066+0000","last_fullsized":"2023-09-12T14:54:59.808066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:09:24.488408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.705573+0000","last_change":"2023-09-12T14:54:58.751371+0000","last_active":"2023-09-12T14:54:59.705573+0000","last_peered":"2023-09-12T14:54:59.705573+0000","last_clean":"2023-09-12T14:54:59.705573+0000","last_became_active":"2023-09-12T14:54:58.750585+0000","last_became_peered":"2023-09-12T14:54:58.750585+0000","last_unstale":"2023-09-12T14:54:59.705573+0000","last_undegraded":"2023-09-12T14:54:59.705573+0000","last_fullsized":"2023-09-12T14:54:59.705573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:18:31.201019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.809302+0000","last_change":"2023-09-12T14:54:58.745970+0000","last_active":"2023-09-12T14:54:59.809302+0000","last_peered":"2023-09-12T14:54:59.809302+0000","last_clean":"2023-09-12T14:54:59.809302+0000","last_became_active":"2023-09-12T14:54:58.745163+0000","last_became_peered":"2023-09-12T14:54:58.745163+0000","last_unstale":"2023-09-12T14:54:59.809302+0000","last_undegraded":"2023-09-12T14:54:59.809302+0000","last_fullsized":"2023-09-12T14:54:59.809302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:55:04.268808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807739+0000","last_change":"2023-09-12T14:54:58.746782+0000","last_active":"2023-09-12T14:54:59.807739+0000","last_peered":"2023-09-12T14:54:59.807739+0000","last_clean":"2023-09-12T14:54:59.807739+0000","last_became_active":"2023-09-12T14:54:58.746149+0000","last_became_peered":"2023-09-12T14:54:58.746149+0000","last_unstale":"2023-09-12T14:54:59.807739+0000","last_undegraded":"2023-09-12T14:54:59.807739+0000","last_fullsized":"2023-09-12T14:54:59.807739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:33:30.961102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.701439+0000","last_change":"2023-09-12T14:54:58.747221+0000","last_active":"2023-09-12T14:54:59.701439+0000","last_peered":"2023-09-12T14:54:59.701439+0000","last_clean":"2023-09-12T14:54:59.701439+0000","last_became_active":"2023-09-12T14:54:58.746509+0000","last_became_peered":"2023-09-12T14:54:58.746509+0000","last_unstale":"2023-09-12T14:54:59.701439+0000","last_undegraded":"2023-09-12T14:54:59.701439+0000","last_fullsized":"2023-09-12T14:54:59.701439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:33.647832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808902+0000","last_change":"2023-09-12T14:54:58.754523+0000","last_active":"2023-09-12T14:54:59.808902+0000","last_peered":"2023-09-12T14:54:59.808902+0000","last_clean":"2023-09-12T14:54:59.808902+0000","last_became_active":"2023-09-12T14:54:58.753849+0000","last_became_peered":"2023-09-12T14:54:58.753849+0000","last_unstale":"2023-09-12T14:54:59.808902+0000","last_undegraded":"2023-09-12T14:54:59.808902+0000","last_fullsized":"2023-09-12T14:54:59.808902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:18:31.201019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806568+0000","last_change":"2023-09-12T14:54:58.750572+0000","last_active":"2023-09-12T14:54:59.806568+0000","last_peered":"2023-09-12T14:54:59.806568+0000","last_clean":"2023-09-12T14:54:59.806568+0000","last_became_active":"2023-09-12T14:54:58.750004+0000","last_became_peered":"2023-09-12T14:54:58.750004+0000","last_unstale":"2023-09-12T14:54:59.806568+0000","last_undegraded":"2023-09-12T14:54:59.806568+0000","last_fullsized":"2023-09-12T14:54:59.806568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:54:14.557968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706258+0000","last_change":"2023-09-12T14:54:58.754746+0000","last_active":"2023-09-12T14:54:59.706258+0000","last_peered":"2023-09-12T14:54:59.706258+0000","last_clean":"2023-09-12T14:54:59.706258+0000","last_became_active":"2023-09-12T14:54:58.754081+0000","last_became_peered":"2023-09-12T14:54:58.754081+0000","last_unstale":"2023-09-12T14:54:59.706258+0000","last_undegraded":"2023-09-12T14:54:59.706258+0000","last_fullsized":"2023-09-12T14:54:59.706258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:23:41.038660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805117+0000","last_change":"2023-09-12T14:54:58.739330+0000","last_active":"2023-09-12T14:54:59.805117+0000","last_peered":"2023-09-12T14:54:59.805117+0000","last_clean":"2023-09-12T14:54:59.805117+0000","last_became_active":"2023-09-12T14:54:58.738451+0000","last_became_peered":"2023-09-12T14:54:58.738451+0000","last_unstale":"2023-09-12T14:54:59.805117+0000","last_undegraded":"2023-09-12T14:54:59.805117+0000","last_fullsized":"2023-09-12T14:54:59.805117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:35.965864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805898+0000","last_change":"2023-09-12T14:54:58.745424+0000","last_active":"2023-09-12T14:54:59.805898+0000","last_peered":"2023-09-12T14:54:59.805898+0000","last_clean":"2023-09-12T14:54:59.805898+0000","last_became_active":"2023-09-12T14:54:58.744842+0000","last_became_peered":"2023-09-12T14:54:58.744842+0000","last_unstale":"2023-09-12T14:54:59.805898+0000","last_undegraded":"2023-09-12T14:54:59.805898+0000","last_fullsized":"2023-09-12T14:54:59.805898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:09:24.488408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.708763+0000","last_change":"2023-09-12T14:54:58.734734+0000","last_active":"2023-09-12T14:54:59.708763+0000","last_peered":"2023-09-12T14:54:59.708763+0000","last_clean":"2023-09-12T14:54:59.708763+0000","last_became_active":"2023-09-12T14:54:58.730411+0000","last_became_peered":"2023-09-12T14:54:58.730411+0000","last_unstale":"2023-09-12T14:54:59.708763+0000","last_undegraded":"2023-09-12T14:54:59.708763+0000","last_fullsized":"2023-09-12T14:54:59.708763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:04:39.183616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808747+0000","last_change":"2023-09-12T14:54:57.709617+0000","last_active":"2023-09-12T14:54:59.808747+0000","last_peered":"2023-09-12T14:54:59.808747+0000","last_clean":"2023-09-12T14:54:59.808747+0000","last_became_active":"2023-09-12T14:54:57.709171+0000","last_became_peered":"2023-09-12T14:54:57.709171+0000","last_unstale":"2023-09-12T14:54:59.808747+0000","last_undegraded":"2023-09-12T14:54:59.808747+0000","last_fullsized":"2023-09-12T14:54:59.808747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:35:05.562961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806849+0000","last_change":"2023-09-12T14:54:58.722571+0000","last_active":"2023-09-12T14:54:59.806849+0000","last_peered":"2023-09-12T14:54:59.806849+0000","last_clean":"2023-09-12T14:54:59.806849+0000","last_became_active":"2023-09-12T14:54:58.721782+0000","last_became_peered":"2023-09-12T14:54:58.721782+0000","last_unstale":"2023-09-12T14:54:59.806849+0000","last_undegraded":"2023-09-12T14:54:59.806849+0000","last_fullsized":"2023-09-12T14:54:59.806849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:22:58.799616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807614+0000","last_change":"2023-09-12T14:54:57.709417+0000","last_active":"2023-09-12T14:54:59.807614+0000","last_peered":"2023-09-12T14:54:59.807614+0000","last_clean":"2023-09-12T14:54:59.807614+0000","last_became_active":"2023-09-12T14:54:57.708756+0000","last_became_peered":"2023-09-12T14:54:57.708756+0000","last_unstale":"2023-09-12T14:54:59.807614+0000","last_undegraded":"2023-09-12T14:54:59.807614+0000","last_fullsized":"2023-09-12T14:54:59.807614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:50:58.241369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808240+0000","last_change":"2023-09-12T14:54:57.698971+0000","last_active":"2023-09-12T14:54:59.808240+0000","last_peered":"2023-09-12T14:54:59.808240+0000","last_clean":"2023-09-12T14:54:59.808240+0000","last_became_active":"2023-09-12T14:54:57.698574+0000","last_became_peered":"2023-09-12T14:54:57.698574+0000","last_unstale":"2023-09-12T14:54:59.808240+0000","last_undegraded":"2023-09-12T14:54:59.808240+0000","last_fullsized":"2023-09-12T14:54:59.808240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:25:01.213694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808091+0000","last_change":"2023-09-12T14:54:58.748244+0000","last_active":"2023-09-12T14:54:59.808091+0000","last_peered":"2023-09-12T14:54:59.808091+0000","last_clean":"2023-09-12T14:54:59.808091+0000","last_became_active":"2023-09-12T14:54:58.746823+0000","last_became_peered":"2023-09-12T14:54:58.746823+0000","last_unstale":"2023-09-12T14:54:59.808091+0000","last_undegraded":"2023-09-12T14:54:59.808091+0000","last_fullsized":"2023-09-12T14:54:59.808091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:22:58.799616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706293+0000","last_change":"2023-09-12T14:54:57.708138+0000","last_active":"2023-09-12T14:54:59.706293+0000","last_peered":"2023-09-12T14:54:59.706293+0000","last_clean":"2023-09-12T14:54:59.706293+0000","last_became_active":"2023-09-12T14:54:57.705380+0000","last_became_peered":"2023-09-12T14:54:57.705380+0000","last_unstale":"2023-09-12T14:54:59.706293+0000","last_undegraded":"2023-09-12T14:54:59.706293+0000","last_fullsized":"2023-09-12T14:54:59.706293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:34:38.380062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.700719+0000","last_change":"2023-09-12T14:54:58.751072+0000","last_active":"2023-09-12T14:54:59.700719+0000","last_peered":"2023-09-12T14:54:59.700719+0000","last_clean":"2023-09-12T14:54:59.700719+0000","last_became_active":"2023-09-12T14:54:58.750309+0000","last_became_peered":"2023-09-12T14:54:58.750309+0000","last_unstale":"2023-09-12T14:54:59.700719+0000","last_undegraded":"2023-09-12T14:54:59.700719+0000","last_fullsized":"2023-09-12T14:54:59.700719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:58:08.195627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808058+0000","last_change":"2023-09-12T14:54:57.719821+0000","last_active":"2023-09-12T14:54:59.808058+0000","last_peered":"2023-09-12T14:54:59.808058+0000","last_clean":"2023-09-12T14:54:59.808058+0000","last_became_active":"2023-09-12T14:54:57.717672+0000","last_became_peered":"2023-09-12T14:54:57.717672+0000","last_unstale":"2023-09-12T14:54:59.808058+0000","last_undegraded":"2023-09-12T14:54:59.808058+0000","last_fullsized":"2023-09-12T14:54:59.808058+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:25:01.213694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.707290+0000","last_change":"2023-09-12T14:54:58.745459+0000","last_active":"2023-09-12T14:54:59.707290+0000","last_peered":"2023-09-12T14:54:59.707290+0000","last_clean":"2023-09-12T14:54:59.707290+0000","last_became_active":"2023-09-12T14:54:58.744478+0000","last_became_peered":"2023-09-12T14:54:58.744478+0000","last_unstale":"2023-09-12T14:54:59.707290+0000","last_undegraded":"2023-09-12T14:54:59.707290+0000","last_fullsized":"2023-09-12T14:54:59.707290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:07:30.650142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.707357+0000","last_change":"2023-09-12T14:54:57.712325+0000","last_active":"2023-09-12T14:54:59.707357+0000","last_peered":"2023-09-12T14:54:59.707357+0000","last_clean":"2023-09-12T14:54:59.707357+0000","last_became_active":"2023-09-12T14:54:57.711669+0000","last_became_peered":"2023-09-12T14:54:57.711669+0000","last_unstale":"2023-09-12T14:54:59.707357+0000","last_undegraded":"2023-09-12T14:54:59.707357+0000","last_fullsized":"2023-09-12T14:54:59.707357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:25:01.213694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807090+0000","last_change":"2023-09-12T14:54:58.731110+0000","last_active":"2023-09-12T14:54:59.807090+0000","last_peered":"2023-09-12T14:54:59.807090+0000","last_clean":"2023-09-12T14:54:59.807090+0000","last_became_active":"2023-09-12T14:54:58.728351+0000","last_became_peered":"2023-09-12T14:54:58.728351+0000","last_unstale":"2023-09-12T14:54:59.807090+0000","last_undegraded":"2023-09-12T14:54:59.807090+0000","last_fullsized":"2023-09-12T14:54:59.807090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:10:09.339305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.700987+0000","last_change":"2023-09-12T14:54:57.699372+0000","last_active":"2023-09-12T14:54:59.700987+0000","last_peered":"2023-09-12T14:54:59.700987+0000","last_clean":"2023-09-12T14:54:59.700987+0000","last_became_active":"2023-09-12T14:54:57.698970+0000","last_became_peered":"2023-09-12T14:54:57.698970+0000","last_unstale":"2023-09-12T14:54:59.700987+0000","last_undegraded":"2023-09-12T14:54:59.700987+0000","last_fullsized":"2023-09-12T14:54:59.700987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:50:58.241369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.709034+0000","last_change":"2023-09-12T14:54:58.753601+0000","last_active":"2023-09-12T14:54:59.709034+0000","last_peered":"2023-09-12T14:54:59.709034+0000","last_clean":"2023-09-12T14:54:59.709034+0000","last_became_active":"2023-09-12T14:54:58.752414+0000","last_became_peered":"2023-09-12T14:54:58.752414+0000","last_unstale":"2023-09-12T14:54:59.709034+0000","last_undegraded":"2023-09-12T14:54:59.709034+0000","last_fullsized":"2023-09-12T14:54:59.709034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:41:25.126595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807970+0000","last_change":"2023-09-12T14:54:57.699723+0000","last_active":"2023-09-12T14:54:59.807970+0000","last_peered":"2023-09-12T14:54:59.807970+0000","last_clean":"2023-09-12T14:54:59.807970+0000","last_became_active":"2023-09-12T14:54:57.698942+0000","last_became_peered":"2023-09-12T14:54:57.698942+0000","last_unstale":"2023-09-12T14:54:59.807970+0000","last_undegraded":"2023-09-12T14:54:59.807970+0000","last_fullsized":"2023-09-12T14:54:59.807970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:35:05.562961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806782+0000","last_change":"2023-09-12T14:54:58.756683+0000","last_active":"2023-09-12T14:54:59.806782+0000","last_peered":"2023-09-12T14:54:59.806782+0000","last_clean":"2023-09-12T14:54:59.806782+0000","last_became_active":"2023-09-12T14:54:58.755952+0000","last_became_peered":"2023-09-12T14:54:58.755952+0000","last_unstale":"2023-09-12T14:54:59.806782+0000","last_undegraded":"2023-09-12T14:54:59.806782+0000","last_fullsized":"2023-09-12T14:54:59.806782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:07:26.691746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806819+0000","last_change":"2023-09-12T14:54:57.708009+0000","last_active":"2023-09-12T14:54:59.806819+0000","last_peered":"2023-09-12T14:54:59.806819+0000","last_clean":"2023-09-12T14:54:59.806819+0000","last_became_active":"2023-09-12T14:54:57.707617+0000","last_became_peered":"2023-09-12T14:54:57.707617+0000","last_unstale":"2023-09-12T14:54:59.806819+0000","last_undegraded":"2023-09-12T14:54:59.806819+0000","last_fullsized":"2023-09-12T14:54:59.806819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:10:06.188199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806973+0000","last_change":"2023-09-12T14:54:58.747555+0000","last_active":"2023-09-12T14:54:59.806973+0000","last_peered":"2023-09-12T14:54:59.806973+0000","last_clean":"2023-09-12T14:54:59.806973+0000","last_became_active":"2023-09-12T14:54:58.745580+0000","last_became_peered":"2023-09-12T14:54:58.745580+0000","last_unstale":"2023-09-12T14:54:59.806973+0000","last_undegraded":"2023-09-12T14:54:59.806973+0000","last_fullsized":"2023-09-12T14:54:59.806973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:33.647832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706246+0000","last_change":"2023-09-12T14:54:57.713557+0000","last_active":"2023-09-12T14:54:59.706246+0000","last_peered":"2023-09-12T14:54:59.706246+0000","last_clean":"2023-09-12T14:54:59.706246+0000","last_became_active":"2023-09-12T14:54:57.712038+0000","last_became_peered":"2023-09-12T14:54:57.712038+0000","last_unstale":"2023-09-12T14:54:59.706246+0000","last_undegraded":"2023-09-12T14:54:59.706246+0000","last_fullsized":"2023-09-12T14:54:59.706246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:22:57.788678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806857+0000","last_change":"2023-09-12T14:54:58.754016+0000","last_active":"2023-09-12T14:54:59.806857+0000","last_peered":"2023-09-12T14:54:59.806857+0000","last_clean":"2023-09-12T14:54:59.806857+0000","last_became_active":"2023-09-12T14:54:58.753380+0000","last_became_peered":"2023-09-12T14:54:58.753380+0000","last_unstale":"2023-09-12T14:54:59.806857+0000","last_undegraded":"2023-09-12T14:54:59.806857+0000","last_fullsized":"2023-09-12T14:54:59.806857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:16:18.487363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805823+0000","last_change":"2023-09-12T14:54:57.713543+0000","last_active":"2023-09-12T14:54:59.805823+0000","last_peered":"2023-09-12T14:54:59.805823+0000","last_clean":"2023-09-12T14:54:59.805823+0000","last_became_active":"2023-09-12T14:54:57.712819+0000","last_became_peered":"2023-09-12T14:54:57.712819+0000","last_unstale":"2023-09-12T14:54:59.805823+0000","last_undegraded":"2023-09-12T14:54:59.805823+0000","last_fullsized":"2023-09-12T14:54:59.805823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:10:06.188199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805969+0000","last_change":"2023-09-12T14:54:58.745224+0000","last_active":"2023-09-12T14:54:59.805969+0000","last_peered":"2023-09-12T14:54:59.805969+0000","last_clean":"2023-09-12T14:54:59.805969+0000","last_became_active":"2023-09-12T14:54:58.744566+0000","last_became_peered":"2023-09-12T14:54:58.744566+0000","last_unstale":"2023-09-12T14:54:59.805969+0000","last_undegraded":"2023-09-12T14:54:59.805969+0000","last_fullsized":"2023-09-12T14:54:59.805969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:56:13.426275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805117+0000","last_change":"2023-09-12T14:54:57.683795+0000","last_active":"2023-09-12T14:54:59.805117+0000","last_peered":"2023-09-12T14:54:59.805117+0000","last_clean":"2023-09-12T14:54:59.805117+0000","last_became_active":"2023-09-12T14:54:57.683309+0000","last_became_peered":"2023-09-12T14:54:57.683309+0000","last_unstale":"2023-09-12T14:54:59.805117+0000","last_undegraded":"2023-09-12T14:54:59.805117+0000","last_fullsized":"2023-09-12T14:54:59.805117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:38:46.256403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806420+0000","last_change":"2023-09-12T14:54:58.744197+0000","last_active":"2023-09-12T14:54:59.806420+0000","last_peered":"2023-09-12T14:54:59.806420+0000","last_clean":"2023-09-12T14:54:59.806420+0000","last_became_active":"2023-09-12T14:54:58.743668+0000","last_became_peered":"2023-09-12T14:54:58.743668+0000","last_unstale":"2023-09-12T14:54:59.806420+0000","last_undegraded":"2023-09-12T14:54:59.806420+0000","last_fullsized":"2023-09-12T14:54:59.806420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:33:30.961102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.709354+0000","last_change":"2023-09-12T14:54:57.704349+0000","last_active":"2023-09-12T14:54:59.709354+0000","last_peered":"2023-09-12T14:54:59.709354+0000","last_clean":"2023-09-12T14:54:59.709354+0000","last_became_active":"2023-09-12T14:54:57.703134+0000","last_became_peered":"2023-09-12T14:54:57.703134+0000","last_unstale":"2023-09-12T14:54:59.709354+0000","last_undegraded":"2023-09-12T14:54:59.709354+0000","last_fullsized":"2023-09-12T14:54:59.709354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:25:54.071276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807199+0000","last_change":"2023-09-12T14:54:58.752753+0000","last_active":"2023-09-12T14:54:59.807199+0000","last_peered":"2023-09-12T14:54:59.807199+0000","last_clean":"2023-09-12T14:54:59.807199+0000","last_became_active":"2023-09-12T14:54:58.751852+0000","last_became_peered":"2023-09-12T14:54:58.751852+0000","last_unstale":"2023-09-12T14:54:59.807199+0000","last_undegraded":"2023-09-12T14:54:59.807199+0000","last_fullsized":"2023-09-12T14:54:59.807199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:23:41.038660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806233+0000","last_change":"2023-09-12T14:54:57.691156+0000","last_active":"2023-09-12T14:54:59.806233+0000","last_peered":"2023-09-12T14:54:59.806233+0000","last_clean":"2023-09-12T14:54:59.806233+0000","last_became_active":"2023-09-12T14:54:57.690769+0000","last_became_peered":"2023-09-12T14:54:57.690769+0000","last_unstale":"2023-09-12T14:54:59.806233+0000","last_undegraded":"2023-09-12T14:54:59.806233+0000","last_fullsized":"2023-09-12T14:54:59.806233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:48:28.111452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807364+0000","last_change":"2023-09-12T14:54:58.755613+0000","last_active":"2023-09-12T14:54:59.807364+0000","last_peered":"2023-09-12T14:54:59.807364+0000","last_clean":"2023-09-12T14:54:59.807364+0000","last_became_active":"2023-09-12T14:54:58.755005+0000","last_became_peered":"2023-09-12T14:54:58.755005+0000","last_unstale":"2023-09-12T14:54:59.807364+0000","last_undegraded":"2023-09-12T14:54:59.807364+0000","last_fullsized":"2023-09-12T14:54:59.807364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:27:30.157653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.701127+0000","last_change":"2023-09-12T14:54:57.686468+0000","last_active":"2023-09-12T14:54:59.701127+0000","last_peered":"2023-09-12T14:54:59.701127+0000","last_clean":"2023-09-12T14:54:59.701127+0000","last_became_active":"2023-09-12T14:54:57.686059+0000","last_became_peered":"2023-09-12T14:54:57.686059+0000","last_unstale":"2023-09-12T14:54:59.701127+0000","last_undegraded":"2023-09-12T14:54:59.701127+0000","last_fullsized":"2023-09-12T14:54:59.701127+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:35:05.562961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706190+0000","last_change":"2023-09-12T14:54:58.751002+0000","last_active":"2023-09-12T14:54:59.706190+0000","last_peered":"2023-09-12T14:54:59.706190+0000","last_clean":"2023-09-12T14:54:59.706190+0000","last_became_active":"2023-09-12T14:54:58.750380+0000","last_became_peered":"2023-09-12T14:54:58.750380+0000","last_unstale":"2023-09-12T14:54:59.706190+0000","last_undegraded":"2023-09-12T14:54:59.706190+0000","last_fullsized":"2023-09-12T14:54:59.706190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:56:32.731297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805597+0000","last_change":"2023-09-12T14:54:57.680566+0000","last_active":"2023-09-12T14:54:59.805597+0000","last_peered":"2023-09-12T14:54:59.805597+0000","last_clean":"2023-09-12T14:54:59.805597+0000","last_became_active":"2023-09-12T14:54:57.680113+0000","last_became_peered":"2023-09-12T14:54:57.680113+0000","last_unstale":"2023-09-12T14:54:59.805597+0000","last_undegraded":"2023-09-12T14:54:59.805597+0000","last_fullsized":"2023-09-12T14:54:59.805597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:48:51.271671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806912+0000","last_change":"2023-09-12T14:54:58.746352+0000","last_active":"2023-09-12T14:54:59.806912+0000","last_peered":"2023-09-12T14:54:59.806912+0000","last_clean":"2023-09-12T14:54:59.806912+0000","last_became_active":"2023-09-12T14:54:58.745469+0000","last_became_peered":"2023-09-12T14:54:58.745469+0000","last_unstale":"2023-09-12T14:54:59.806912+0000","last_undegraded":"2023-09-12T14:54:59.806912+0000","last_fullsized":"2023-09-12T14:54:59.806912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:56:32.731297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-12T14:54:58.722614+0000","last_change":"2023-09-12T14:54:46.965446+0000","last_active":"2023-09-12T14:54:58.722614+0000","last_peered":"2023-09-12T14:54:58.722614+0000","last_clean":"2023-09-12T14:54:58.722614+0000","last_became_active":"2023-09-12T14:54:46.963236+0000","last_became_peered":"2023-09-12T14:54:46.963236+0000","last_unstale":"2023-09-12T14:54:58.722614+0000","last_undegraded":"2023-09-12T14:54:58.722614+0000","last_fullsized":"2023-09-12T14:54:58.722614+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:45.804232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:45.804232+0000","last_clean_scrub_stamp":"2023-09-12T14:54:45.804232+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:24:45.670513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.705459+0000","last_change":"2023-09-12T14:54:57.707645+0000","last_active":"2023-09-12T14:54:59.705459+0000","last_peered":"2023-09-12T14:54:59.705459+0000","last_clean":"2023-09-12T14:54:59.705459+0000","last_became_active":"2023-09-12T14:54:57.705665+0000","last_became_peered":"2023-09-12T14:54:57.705665+0000","last_unstale":"2023-09-12T14:54:59.705459+0000","last_undegraded":"2023-09-12T14:54:59.705459+0000","last_fullsized":"2023-09-12T14:54:59.705459+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:48:28.111452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808625+0000","last_change":"2023-09-12T14:54:58.745179+0000","last_active":"2023-09-12T14:54:59.808625+0000","last_peered":"2023-09-12T14:54:59.808625+0000","last_clean":"2023-09-12T14:54:59.808625+0000","last_became_active":"2023-09-12T14:54:58.744548+0000","last_became_peered":"2023-09-12T14:54:58.744548+0000","last_unstale":"2023-09-12T14:54:59.808625+0000","last_undegraded":"2023-09-12T14:54:59.808625+0000","last_fullsized":"2023-09-12T14:54:59.808625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:10:09.339305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806036+0000","last_change":"2023-09-12T14:54:57.719515+0000","last_active":"2023-09-12T14:54:59.806036+0000","last_peered":"2023-09-12T14:54:59.806036+0000","last_clean":"2023-09-12T14:54:59.806036+0000","last_became_active":"2023-09-12T14:54:57.717582+0000","last_became_peered":"2023-09-12T14:54:57.717582+0000","last_unstale":"2023-09-12T14:54:59.806036+0000","last_undegraded":"2023-09-12T14:54:59.806036+0000","last_fullsized":"2023-09-12T14:54:59.806036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:48:51.271671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806471+0000","last_change":"2023-09-12T14:54:58.715137+0000","last_active":"2023-09-12T14:54:59.806471+0000","last_peered":"2023-09-12T14:54:59.806471+0000","last_clean":"2023-09-12T14:54:59.806471+0000","last_became_active":"2023-09-12T14:54:58.713883+0000","last_became_peered":"2023-09-12T14:54:58.713883+0000","last_unstale":"2023-09-12T14:54:59.806471+0000","last_undegraded":"2023-09-12T14:54:59.806471+0000","last_fullsized":"2023-09-12T14:54:59.806471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:20:59.118588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805239+0000","last_change":"2023-09-12T14:54:57.691863+0000","last_active":"2023-09-12T14:54:59.805239+0000","last_peered":"2023-09-12T14:54:59.805239+0000","last_clean":"2023-09-12T14:54:59.805239+0000","last_became_active":"2023-09-12T14:54:57.691453+0000","last_became_peered":"2023-09-12T14:54:57.691453+0000","last_unstale":"2023-09-12T14:54:59.805239+0000","last_undegraded":"2023-09-12T14:54:59.805239+0000","last_fullsized":"2023-09-12T14:54:59.805239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:01:34.927854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.700716+0000","last_change":"2023-09-12T14:54:58.732432+0000","last_active":"2023-09-12T14:54:59.700716+0000","last_peered":"2023-09-12T14:54:59.700716+0000","last_clean":"2023-09-12T14:54:59.700716+0000","last_became_active":"2023-09-12T14:54:58.731662+0000","last_became_peered":"2023-09-12T14:54:58.731662+0000","last_unstale":"2023-09-12T14:54:59.700716+0000","last_undegraded":"2023-09-12T14:54:59.700716+0000","last_fullsized":"2023-09-12T14:54:59.700716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:09:24.488408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706495+0000","last_change":"2023-09-12T14:54:57.714059+0000","last_active":"2023-09-12T14:54:59.706495+0000","last_peered":"2023-09-12T14:54:59.706495+0000","last_clean":"2023-09-12T14:54:59.706495+0000","last_became_active":"2023-09-12T14:54:57.712854+0000","last_became_peered":"2023-09-12T14:54:57.712854+0000","last_unstale":"2023-09-12T14:54:59.706495+0000","last_undegraded":"2023-09-12T14:54:59.706495+0000","last_fullsized":"2023-09-12T14:54:59.706495+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:56:12.415337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.709032+0000","last_change":"2023-09-12T14:54:58.713195+0000","last_active":"2023-09-12T14:54:59.709032+0000","last_peered":"2023-09-12T14:54:59.709032+0000","last_clean":"2023-09-12T14:54:59.709032+0000","last_became_active":"2023-09-12T14:54:58.712392+0000","last_became_peered":"2023-09-12T14:54:58.712392+0000","last_unstale":"2023-09-12T14:54:59.709032+0000","last_undegraded":"2023-09-12T14:54:59.709032+0000","last_fullsized":"2023-09-12T14:54:59.709032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:01:35.938792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.701021+0000","last_change":"2023-09-12T14:54:57.691816+0000","last_active":"2023-09-12T14:54:59.701021+0000","last_peered":"2023-09-12T14:54:59.701021+0000","last_clean":"2023-09-12T14:54:59.701021+0000","last_became_active":"2023-09-12T14:54:57.691410+0000","last_became_peered":"2023-09-12T14:54:57.691410+0000","last_unstale":"2023-09-12T14:54:59.701021+0000","last_undegraded":"2023-09-12T14:54:59.701021+0000","last_fullsized":"2023-09-12T14:54:59.701021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:01:34.927854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706911+0000","last_change":"2023-09-12T14:54:58.748697+0000","last_active":"2023-09-12T14:54:59.706911+0000","last_peered":"2023-09-12T14:54:59.706911+0000","last_clean":"2023-09-12T14:54:59.706911+0000","last_became_active":"2023-09-12T14:54:58.748072+0000","last_became_peered":"2023-09-12T14:54:58.748072+0000","last_unstale":"2023-09-12T14:54:59.706911+0000","last_undegraded":"2023-09-12T14:54:59.706911+0000","last_fullsized":"2023-09-12T14:54:59.706911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:27:30.157653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807241+0000","last_change":"2023-09-12T14:54:57.718970+0000","last_active":"2023-09-12T14:54:59.807241+0000","last_peered":"2023-09-12T14:54:59.807241+0000","last_clean":"2023-09-12T14:54:59.807241+0000","last_became_active":"2023-09-12T14:54:57.717410+0000","last_became_peered":"2023-09-12T14:54:57.717410+0000","last_unstale":"2023-09-12T14:54:59.807241+0000","last_undegraded":"2023-09-12T14:54:59.807241+0000","last_fullsized":"2023-09-12T14:54:59.807241+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:22:57.788678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.701025+0000","last_change":"2023-09-12T14:54:58.733797+0000","last_active":"2023-09-12T14:54:59.701025+0000","last_peered":"2023-09-12T14:54:59.701025+0000","last_clean":"2023-09-12T14:54:59.701025+0000","last_became_active":"2023-09-12T14:54:58.732869+0000","last_became_peered":"2023-09-12T14:54:58.732869+0000","last_unstale":"2023-09-12T14:54:59.701025+0000","last_undegraded":"2023-09-12T14:54:59.701025+0000","last_fullsized":"2023-09-12T14:54:59.701025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:22:58.799616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:55:00.614576+0000","last_change":"2023-09-12T14:54:57.703103+0000","last_active":"2023-09-12T14:55:00.614576+0000","last_peered":"2023-09-12T14:55:00.614576+0000","last_clean":"2023-09-12T14:55:00.614576+0000","last_became_active":"2023-09-12T14:54:57.702706+0000","last_became_peered":"2023-09-12T14:54:57.702706+0000","last_unstale":"2023-09-12T14:55:00.614576+0000","last_undegraded":"2023-09-12T14:55:00.614576+0000","last_fullsized":"2023-09-12T14:55:00.614576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:34:38.380062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.809481+0000","last_change":"2023-09-12T14:54:58.745944+0000","last_active":"2023-09-12T14:54:59.809481+0000","last_peered":"2023-09-12T14:54:59.809481+0000","last_clean":"2023-09-12T14:54:59.809481+0000","last_became_active":"2023-09-12T14:54:58.744677+0000","last_became_peered":"2023-09-12T14:54:58.744677+0000","last_unstale":"2023-09-12T14:54:59.809481+0000","last_undegraded":"2023-09-12T14:54:59.809481+0000","last_fullsized":"2023-09-12T14:54:59.809481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:26:45.799536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808147+0000","last_change":"2023-09-12T14:54:57.719402+0000","last_active":"2023-09-12T14:54:59.808147+0000","last_peered":"2023-09-12T14:54:59.808147+0000","last_clean":"2023-09-12T14:54:59.808147+0000","last_became_active":"2023-09-12T14:54:57.718815+0000","last_became_peered":"2023-09-12T14:54:57.718815+0000","last_unstale":"2023-09-12T14:54:59.808147+0000","last_undegraded":"2023-09-12T14:54:59.808147+0000","last_fullsized":"2023-09-12T14:54:59.808147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:20:58.107650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807282+0000","last_change":"2023-09-12T14:54:58.732619+0000","last_active":"2023-09-12T14:54:59.807282+0000","last_peered":"2023-09-12T14:54:59.807282+0000","last_clean":"2023-09-12T14:54:59.807282+0000","last_became_active":"2023-09-12T14:54:58.731845+0000","last_became_peered":"2023-09-12T14:54:58.731845+0000","last_unstale":"2023-09-12T14:54:59.807282+0000","last_undegraded":"2023-09-12T14:54:59.807282+0000","last_fullsized":"2023-09-12T14:54:59.807282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:33:30.961102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805781+0000","last_change":"2023-09-12T14:54:57.717947+0000","last_active":"2023-09-12T14:54:59.805781+0000","last_peered":"2023-09-12T14:54:59.805781+0000","last_clean":"2023-09-12T14:54:59.805781+0000","last_became_active":"2023-09-12T14:54:57.716465+0000","last_became_peered":"2023-09-12T14:54:57.716465+0000","last_unstale":"2023-09-12T14:54:59.805781+0000","last_undegraded":"2023-09-12T14:54:59.805781+0000","last_fullsized":"2023-09-12T14:54:59.805781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:01:34.927854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.708729+0000","last_change":"2023-09-12T14:54:58.734846+0000","last_active":"2023-09-12T14:54:59.708729+0000","last_peered":"2023-09-12T14:54:59.708729+0000","last_clean":"2023-09-12T14:54:59.708729+0000","last_became_active":"2023-09-12T14:54:58.733232+0000","last_became_peered":"2023-09-12T14:54:58.733232+0000","last_unstale":"2023-09-12T14:54:59.708729+0000","last_undegraded":"2023-09-12T14:54:59.708729+0000","last_fullsized":"2023-09-12T14:54:59.708729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:48:52.282609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.700759+0000","last_change":"2023-09-12T14:54:57.693075+0000","last_active":"2023-09-12T14:54:59.700759+0000","last_peered":"2023-09-12T14:54:59.700759+0000","last_clean":"2023-09-12T14:54:59.700759+0000","last_became_active":"2023-09-12T14:54:57.692190+0000","last_became_peered":"2023-09-12T14:54:57.692190+0000","last_unstale":"2023-09-12T14:54:59.700759+0000","last_undegraded":"2023-09-12T14:54:59.700759+0000","last_fullsized":"2023-09-12T14:54:59.700759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:41:24.115657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808265+0000","last_change":"2023-09-12T14:54:58.748674+0000","last_active":"2023-09-12T14:54:59.808265+0000","last_peered":"2023-09-12T14:54:59.808265+0000","last_clean":"2023-09-12T14:54:59.808265+0000","last_became_active":"2023-09-12T14:54:58.747570+0000","last_became_peered":"2023-09-12T14:54:58.747570+0000","last_unstale":"2023-09-12T14:54:59.808265+0000","last_undegraded":"2023-09-12T14:54:59.808265+0000","last_fullsized":"2023-09-12T14:54:59.808265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:56:13.426275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.700971+0000","last_change":"2023-09-12T14:54:57.690103+0000","last_active":"2023-09-12T14:54:59.700971+0000","last_peered":"2023-09-12T14:54:59.700971+0000","last_clean":"2023-09-12T14:54:59.700971+0000","last_became_active":"2023-09-12T14:54:57.688945+0000","last_became_peered":"2023-09-12T14:54:57.688945+0000","last_unstale":"2023-09-12T14:54:59.700971+0000","last_undegraded":"2023-09-12T14:54:59.700971+0000","last_fullsized":"2023-09-12T14:54:59.700971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:25:01.213694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.808786+0000","last_change":"2023-09-12T14:54:58.744905+0000","last_active":"2023-09-12T14:54:59.808786+0000","last_peered":"2023-09-12T14:54:59.808786+0000","last_clean":"2023-09-12T14:54:59.808786+0000","last_became_active":"2023-09-12T14:54:58.744132+0000","last_became_peered":"2023-09-12T14:54:58.744132+0000","last_unstale":"2023-09-12T14:54:59.808786+0000","last_undegraded":"2023-09-12T14:54:59.808786+0000","last_fullsized":"2023-09-12T14:54:59.808786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:33:30.961102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807246+0000","last_change":"2023-09-12T14:54:57.718508+0000","last_active":"2023-09-12T14:54:59.807246+0000","last_peered":"2023-09-12T14:54:59.807246+0000","last_clean":"2023-09-12T14:54:59.807246+0000","last_became_active":"2023-09-12T14:54:57.716751+0000","last_became_peered":"2023-09-12T14:54:57.716751+0000","last_unstale":"2023-09-12T14:54:59.807246+0000","last_undegraded":"2023-09-12T14:54:59.807246+0000","last_fullsized":"2023-09-12T14:54:59.807246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:34:38.380062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.805940+0000","last_change":"2023-09-12T14:54:58.747254+0000","last_active":"2023-09-12T14:54:59.805940+0000","last_peered":"2023-09-12T14:54:59.805940+0000","last_clean":"2023-09-12T14:54:59.805940+0000","last_became_active":"2023-09-12T14:54:58.746726+0000","last_became_peered":"2023-09-12T14:54:58.746726+0000","last_unstale":"2023-09-12T14:54:59.805940+0000","last_undegraded":"2023-09-12T14:54:59.805940+0000","last_fullsized":"2023-09-12T14:54:59.805940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:22:58.799616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.706879+0000","last_change":"2023-09-12T14:54:58.720140+0000","last_active":"2023-09-12T14:54:59.706879+0000","last_peered":"2023-09-12T14:54:59.706879+0000","last_clean":"2023-09-12T14:54:59.706879+0000","last_became_active":"2023-09-12T14:54:58.717751+0000","last_became_peered":"2023-09-12T14:54:58.717751+0000","last_unstale":"2023-09-12T14:54:59.706879+0000","last_undegraded":"2023-09-12T14:54:59.706879+0000","last_fullsized":"2023-09-12T14:54:59.706879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:13:39.293184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806375+0000","last_change":"2023-09-12T14:54:57.717851+0000","last_active":"2023-09-12T14:54:59.806375+0000","last_peered":"2023-09-12T14:54:59.806375+0000","last_clean":"2023-09-12T14:54:59.806375+0000","last_became_active":"2023-09-12T14:54:57.716821+0000","last_became_peered":"2023-09-12T14:54:57.716821+0000","last_unstale":"2023-09-12T14:54:59.806375+0000","last_undegraded":"2023-09-12T14:54:59.806375+0000","last_fullsized":"2023-09-12T14:54:59.806375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:59:34.954926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.705907+0000","last_change":"2023-09-12T14:54:58.722319+0000","last_active":"2023-09-12T14:54:59.705907+0000","last_peered":"2023-09-12T14:54:59.705907+0000","last_clean":"2023-09-12T14:54:59.705907+0000","last_became_active":"2023-09-12T14:54:58.719871+0000","last_became_peered":"2023-09-12T14:54:58.719871+0000","last_unstale":"2023-09-12T14:54:59.705907+0000","last_undegraded":"2023-09-12T14:54:59.705907+0000","last_fullsized":"2023-09-12T14:54:59.705907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:07:26.691746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807030+0000","last_change":"2023-09-12T14:54:57.718179+0000","last_active":"2023-09-12T14:54:59.807030+0000","last_peered":"2023-09-12T14:54:59.807030+0000","last_clean":"2023-09-12T14:54:59.807030+0000","last_became_active":"2023-09-12T14:54:57.716712+0000","last_became_peered":"2023-09-12T14:54:57.716712+0000","last_unstale":"2023-09-12T14:54:59.807030+0000","last_undegraded":"2023-09-12T14:54:59.807030+0000","last_fullsized":"2023-09-12T14:54:59.807030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:25:01.213694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.806750+0000","last_change":"2023-09-12T14:54:58.728727+0000","last_active":"2023-09-12T14:54:59.806750+0000","last_peered":"2023-09-12T14:54:59.806750+0000","last_clean":"2023-09-12T14:54:59.806750+0000","last_became_active":"2023-09-12T14:54:58.727977+0000","last_became_peered":"2023-09-12T14:54:58.727977+0000","last_unstale":"2023-09-12T14:54:59.806750+0000","last_undegraded":"2023-09-12T14:54:59.806750+0000","last_fullsized":"2023-09-12T14:54:59.806750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:09:24.488408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.809447+0000","last_change":"2023-09-12T14:54:57.683980+0000","last_active":"2023-09-12T14:54:59.809447+0000","last_peered":"2023-09-12T14:54:59.809447+0000","last_clean":"2023-09-12T14:54:59.809447+0000","last_became_active":"2023-09-12T14:54:57.683572+0000","last_became_peered":"2023-09-12T14:54:57.683572+0000","last_unstale":"2023-09-12T14:54:59.809447+0000","last_undegraded":"2023-09-12T14:54:59.809447+0000","last_fullsized":"2023-09-12T14:54:59.809447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:10:06.188199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.809386+0000","last_change":"2023-09-12T14:54:58.758048+0000","last_active":"2023-09-12T14:54:59.809386+0000","last_peered":"2023-09-12T14:54:59.809386+0000","last_clean":"2023-09-12T14:54:59.809386+0000","last_became_active":"2023-09-12T14:54:58.757449+0000","last_became_peered":"2023-09-12T14:54:58.757449+0000","last_unstale":"2023-09-12T14:54:59.809386+0000","last_undegraded":"2023-09-12T14:54:59.809386+0000","last_fullsized":"2023-09-12T14:54:59.809386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:57.655859+0000","last_clean_scrub_stamp":"2023-09-12T14:54:57.655859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:07:26.691746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-12T14:54:59.807331+0000","last_change":"2023-09-12T14:54:57.709447+0000","last_active":"2023-09-12T14:54:59.807331+0000","last_peered":"2023-09-12T14:54:59.807331+0000","last_clean":"2023-09-12T14:54:59.807331+0000","last_became_active":"2023-09-12T14:54:57.709126+0000","last_became_peered":"2023-09-12T14:54:57.709126+0000","last_unstale":"2023-09-12T14:54:59.807331+0000","last_undegraded":"2023-09-12T14:54:59.807331+0000","last_fullsized":"2023-09-12T14:54:59.807331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:56.644921+0000","last_clean_scrub_stamp":"2023-09-12T14:54:56.644921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:34:38.380062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4239999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":70988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":806912,"data_stored":660885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-12T14:55:06.819 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-09-12T14:55:06.819 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-09-12T14:55:06.819 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-09-12T14:55:06.819 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-12T14:55:06.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.973+0000 7f4793fcf640 1 -- 172.21.15.131:0/2199942878 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f478c157bc0 msgr2=0x7f478c070a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.973+0000 7f4793fcf640 1 --2- 172.21.15.131:0/2199942878 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f478c157bc0 0x7f478c070a80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4784009900 tx=0x7f478401c830 comp rx=0 tx=0).stop 2023-09-12T14:55:06.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.973+0000 7f4793fcf640 1 -- 172.21.15.131:0/2199942878 shutdown_connections 2023-09-12T14:55:06.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.973+0000 7f4793fcf640 1 --2- 172.21.15.131:0/2199942878 >> v2:172.21.15.187:3300/0 conn(0x7f478c0afa40 0x7f478c0afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.973+0000 7f4793fcf640 1 --2- 172.21.15.131:0/2199942878 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f478c157bc0 0x7f478c070a80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.973+0000 7f4793fcf640 1 -- 172.21.15.131:0/2199942878 >> 172.21.15.131:0/2199942878 conn(0x7f478c0cdd80 msgr2=0x7f478c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:06.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.973+0000 7f4793fcf640 1 -- 172.21.15.131:0/2199942878 shutdown_connections 2023-09-12T14:55:06.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.973+0000 7f4793fcf640 1 -- 172.21.15.131:0/2199942878 wait complete. 2023-09-12T14:55:06.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.977+0000 7f4793fcf640 1 Processor -- start 2023-09-12T14:55:06.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.977+0000 7f4793fcf640 1 -- start start 2023-09-12T14:55:06.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.977+0000 7f4793fcf640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f478c06db90 0x7f478c160e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:06.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.977+0000 7f4793fcf640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f478c0afa40 0x7f478c143610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:06.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.977+0000 7f4793fcf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478c157bc0 0x7f478c14a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:06.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.977+0000 7f4793fcf640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c164140 con 0x7f478c157bc0 2023-09-12T14:55:06.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.977+0000 7f4793fcf640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c161370 con 0x7f478c0afa40 2023-09-12T14:55:06.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.977+0000 7f4793fcf640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f478c143b50 con 0x7f478c06db90 2023-09-12T14:55:06.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.977+0000 7f479256c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478c157bc0 0x7f478c14a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:06.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.977+0000 7f479256c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478c157bc0 0x7f478c14a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39268/0 (socket says 172.21.15.131:39268) 2023-09-12T14:55:06.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.977+0000 7f479256c640 1 -- 172.21.15.131:0/1256553385 learned_addr learned my addr 172.21.15.131:0/1256553385 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:06.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.977+0000 7f479156a640 1 --2- 172.21.15.131:0/1256553385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f478c0afa40 0x7f478c143610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:06.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.977+0000 7f4791d6b640 1 --2- 172.21.15.131:0/1256553385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f478c06db90 0x7f478c160e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:06.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.977+0000 7f479256c640 1 -- 172.21.15.131:0/1256553385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f478c06db90 msgr2=0x7f478c160e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.977+0000 7f479256c640 1 --2- 172.21.15.131:0/1256553385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f478c06db90 0x7f478c160e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.977+0000 7f479256c640 1 -- 172.21.15.131:0/1256553385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f478c0afa40 msgr2=0x7f478c143610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:06.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.977+0000 7f479256c640 1 --2- 172.21.15.131:0/1256553385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f478c0afa40 0x7f478c143610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:06.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.977+0000 7f479256c640 1 -- 172.21.15.131:0/1256553385 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c009820 con 0x7f478c157bc0 2023-09-12T14:55:06.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.977+0000 7f4791d6b640 1 --2- 172.21.15.131:0/1256553385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f478c06db90 0x7f478c160e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:06.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.977+0000 7f479256c640 1 --2- 172.21.15.131:0/1256553385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478c157bc0 0x7f478c14a410 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f477c009d30 tx=0x7f477c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:06.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.977+0000 7f478a7fc640 1 -- 172.21.15.131:0/1256553385 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c009b80 con 0x7f478c157bc0 2023-09-12T14:55:06.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.977+0000 7f478a7fc640 1 -- 172.21.15.131:0/1256553385 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f477c003b80 con 0x7f478c157bc0 2023-09-12T14:55:06.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.977+0000 7f478a7fc640 1 -- 172.21.15.131:0/1256553385 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c004020 con 0x7f478c157bc0 2023-09-12T14:55:06.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.977+0000 7f4793fcf640 1 -- 172.21.15.131:0/1256553385 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4784009560 con 0x7f478c157bc0 2023-09-12T14:55:06.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.977+0000 7f4793fcf640 1 -- 172.21.15.131:0/1256553385 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f478c145d10 con 0x7f478c157bc0 2023-09-12T14:55:06.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.977+0000 7f4793fcf640 1 -- 172.21.15.131:0/1256553385 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f478c073580 con 0x7f478c157bc0 2023-09-12T14:55:06.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.977+0000 7f478a7fc640 1 -- 172.21.15.131:0/1256553385 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f477c004180 con 0x7f478c157bc0 2023-09-12T14:55:06.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.981+0000 7f478a7fc640 1 --2- 172.21.15.131:0/1256553385 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4778065b20 0x7f4778067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:06.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.981+0000 7f478a7fc640 1 -- 172.21.15.131:0/1256553385 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f477c014070 con 0x7f478c157bc0 2023-09-12T14:55:06.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.981+0000 7f4791d6b640 1 --2- 172.21.15.131:0/1256553385 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4778065b20 0x7f4778067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:06.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.981+0000 7f4791d6b640 1 --2- 172.21.15.131:0/1256553385 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4778065b20 0x7f4778067fe0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f47840af000 tx=0x7f478401b070 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:06.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:06.985+0000 7f478a7fc640 1 -- 172.21.15.131:0/1256553385 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f477c051be0 con 0x7f478c157bc0 2023-09-12T14:55:07.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.153+0000 7f4793fcf640 1 -- 172.21.15.131:0/1256553385 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f478c0ad800 con 0x7f478c157bc0 2023-09-12T14:55:07.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.157+0000 7f478a7fc640 1 -- 172.21.15.131:0/1256553385 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f477c055200 con 0x7f478c157bc0 2023-09-12T14:55:07.158 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:07.158 INFO:teuthology.orchestra.run.smithi131.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-09-12T14:55:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.157+0000 7f4793fcf640 1 -- 172.21.15.131:0/1256553385 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4778065b20 msgr2=0x7f4778067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.157+0000 7f4793fcf640 1 --2- 172.21.15.131:0/1256553385 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4778065b20 0x7f4778067fe0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f47840af000 tx=0x7f478401b070 comp rx=0 tx=0).stop 2023-09-12T14:55:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.157+0000 7f4793fcf640 1 -- 172.21.15.131:0/1256553385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478c157bc0 msgr2=0x7f478c14a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:07.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.157+0000 7f4793fcf640 1 --2- 172.21.15.131:0/1256553385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478c157bc0 0x7f478c14a410 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f477c009d30 tx=0x7f477c00c6a0 comp rx=0 tx=0).stop 2023-09-12T14:55:07.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.157+0000 7f4793fcf640 1 -- 172.21.15.131:0/1256553385 shutdown_connections 2023-09-12T14:55:07.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.157+0000 7f4793fcf640 1 --2- 172.21.15.131:0/1256553385 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4778065b20 0x7f4778067fe0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:07.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.157+0000 7f4793fcf640 1 --2- 172.21.15.131:0/1256553385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478c157bc0 0x7f478c14a410 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:07.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.157+0000 7f4793fcf640 1 --2- 172.21.15.131:0/1256553385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f478c0afa40 0x7f478c143610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:07.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.157+0000 7f4793fcf640 1 --2- 172.21.15.131:0/1256553385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f478c06db90 0x7f478c160e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:07.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.157+0000 7f4793fcf640 1 -- 172.21.15.131:0/1256553385 >> 172.21.15.131:0/1256553385 conn(0x7f478c0cdd80 msgr2=0x7f478c0b2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:07.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.157+0000 7f4793fcf640 1 -- 172.21.15.131:0/1256553385 shutdown_connections 2023-09-12T14:55:07.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.157+0000 7f4793fcf640 1 -- 172.21.15.131:0/1256553385 wait complete. 2023-09-12T14:55:07.175 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-09-12T14:55:07.176 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:55:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.285+0000 7fbaa202b640 1 -- 172.21.15.131:0/2798320218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba9c0c9a20 msgr2=0x7fba9c06b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.285+0000 7fbaa202b640 1 --2- 172.21.15.131:0/2798320218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba9c0c9a20 0x7fba9c06b430 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fba9c06eaf0 tx=0x7fba9001c7d0 comp rx=0 tx=0).stop 2023-09-12T14:55:07.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.285+0000 7fbaa202b640 1 -- 172.21.15.131:0/2798320218 shutdown_connections 2023-09-12T14:55:07.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.285+0000 7fbaa202b640 1 --2- 172.21.15.131:0/2798320218 >> v2:172.21.15.187:3300/0 conn(0x7fba9c0abae0 0x7fba9c15ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:07.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.285+0000 7fbaa202b640 1 --2- 172.21.15.131:0/2798320218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba9c0c9a20 0x7fba9c06b430 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:07.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.285+0000 7fbaa202b640 1 -- 172.21.15.131:0/2798320218 >> 172.21.15.131:0/2798320218 conn(0x7fba9c069840 msgr2=0x7fba9c0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:07.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.285+0000 7fbaa202b640 1 -- 172.21.15.131:0/2798320218 shutdown_connections 2023-09-12T14:55:07.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.285+0000 7fbaa202b640 1 -- 172.21.15.131:0/2798320218 wait complete. 2023-09-12T14:55:07.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fbaa202b640 1 Processor -- start 2023-09-12T14:55:07.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fbaa202b640 1 -- start start 2023-09-12T14:55:07.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fbaa202b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba9c0abae0 0x7fba9c142bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:07.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fbaa202b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba9c0c9a20 0x7fba9c140ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:07.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fbaa202b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba9c158d70 0x7fba9c141670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:07.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fbaa202b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c142a40 con 0x7fba9c158d70 2023-09-12T14:55:07.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fbaa202b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c152990 con 0x7fba9c0abae0 2023-09-12T14:55:07.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fbaa202b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c141ec0 con 0x7fba9c0c9a20 2023-09-12T14:55:07.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fba9bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba9c158d70 0x7fba9c141670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:07.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fba9bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba9c158d70 0x7fba9c141670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39290/0 (socket says 172.21.15.131:39290) 2023-09-12T14:55:07.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fba9bfff640 1 -- 172.21.15.131:0/4041398959 learned_addr learned my addr 172.21.15.131:0/4041398959 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:07.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fba9b7fe640 1 --2- 172.21.15.131:0/4041398959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba9c0abae0 0x7fba9c142bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:07.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fba9affd640 1 --2- 172.21.15.131:0/4041398959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba9c0c9a20 0x7fba9c140ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:07.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fba9bfff640 1 -- 172.21.15.131:0/4041398959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba9c0c9a20 msgr2=0x7fba9c140ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:07.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fba9bfff640 1 --2- 172.21.15.131:0/4041398959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba9c0c9a20 0x7fba9c140ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:07.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fba9bfff640 1 -- 172.21.15.131:0/4041398959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba9c0abae0 msgr2=0x7fba9c142bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:07.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fba9bfff640 1 --2- 172.21.15.131:0/4041398959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba9c0abae0 0x7fba9c142bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:07.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fba9bfff640 1 -- 172.21.15.131:0/4041398959 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba90009560 con 0x7fba9c158d70 2023-09-12T14:55:07.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fba9affd640 1 --2- 172.21.15.131:0/4041398959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba9c0c9a20 0x7fba9c140ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:07.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fba9bfff640 1 --2- 172.21.15.131:0/4041398959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba9c158d70 0x7fba9c141670 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fba9400b9e0 tx=0x7fba9400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:07.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fba83fff640 1 -- 172.21.15.131:0/4041398959 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9400cb20 con 0x7fba9c158d70 2023-09-12T14:55:07.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fba83fff640 1 -- 172.21.15.131:0/4041398959 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fba9400cd90 con 0x7fba9c158d70 2023-09-12T14:55:07.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fbaa202b640 1 -- 172.21.15.131:0/4041398959 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba9c1421a0 con 0x7fba9c158d70 2023-09-12T14:55:07.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fba83fff640 1 -- 172.21.15.131:0/4041398959 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba940146a0 con 0x7fba9c158d70 2023-09-12T14:55:07.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fbaa202b640 1 -- 172.21.15.131:0/4041398959 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba9c142760 con 0x7fba9c158d70 2023-09-12T14:55:07.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fbaa202b640 1 -- 172.21.15.131:0/4041398959 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba9c073580 con 0x7fba9c158d70 2023-09-12T14:55:07.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fba83fff640 1 -- 172.21.15.131:0/4041398959 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fba9401d3b0 con 0x7fba9c158d70 2023-09-12T14:55:07.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fba83fff640 1 --2- 172.21.15.131:0/4041398959 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fba88065b20 0x7fba88067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:07.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fba83fff640 1 -- 172.21.15.131:0/4041398959 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba94083510 con 0x7fba9c158d70 2023-09-12T14:55:07.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.289+0000 7fba9b7fe640 1 --2- 172.21.15.131:0/4041398959 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fba88065b20 0x7fba88067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:07.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.293+0000 7fba9b7fe640 1 --2- 172.21.15.131:0/4041398959 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fba88065b20 0x7fba88067fe0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fba9c154d40 tx=0x7fba8c008040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:07.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.293+0000 7fba83fff640 1 -- 172.21.15.131:0/4041398959 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fba94050de0 con 0x7fba9c158d70 2023-09-12T14:55:07.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.437+0000 7fbaa202b640 1 -- 172.21.15.131:0/4041398959 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba9c0cd4d0 con 0x7fba9c158d70 2023-09-12T14:55:07.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.441+0000 7fba83fff640 1 -- 172.21.15.131:0/4041398959 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fba94081e60 con 0x7fba9c158d70 2023-09-12T14:55:07.482 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:07.483 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:54:59.694778+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:54:55.689983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T14:54:57.029447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:55:07.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.441+0000 7fbaa202b640 1 -- 172.21.15.131:0/4041398959 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fba88065b20 msgr2=0x7fba88067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:07.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.441+0000 7fbaa202b640 1 --2- 172.21.15.131:0/4041398959 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fba88065b20 0x7fba88067fe0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fba9c154d40 tx=0x7fba8c008040 comp rx=0 tx=0).stop 2023-09-12T14:55:07.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.441+0000 7fbaa202b640 1 -- 172.21.15.131:0/4041398959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba9c158d70 msgr2=0x7fba9c141670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:07.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.441+0000 7fbaa202b640 1 --2- 172.21.15.131:0/4041398959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba9c158d70 0x7fba9c141670 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fba9400b9e0 tx=0x7fba9400bea0 comp rx=0 tx=0).stop 2023-09-12T14:55:07.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.441+0000 7fbaa202b640 1 -- 172.21.15.131:0/4041398959 shutdown_connections 2023-09-12T14:55:07.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.441+0000 7fbaa202b640 1 --2- 172.21.15.131:0/4041398959 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fba88065b20 0x7fba88067fe0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:07.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.441+0000 7fbaa202b640 1 --2- 172.21.15.131:0/4041398959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba9c0c9a20 0x7fba9c140ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:07.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.441+0000 7fbaa202b640 1 --2- 172.21.15.131:0/4041398959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba9c158d70 0x7fba9c141670 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:07.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.441+0000 7fbaa202b640 1 --2- 172.21.15.131:0/4041398959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba9c0abae0 0x7fba9c142bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:07.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.441+0000 7fbaa202b640 1 -- 172.21.15.131:0/4041398959 >> 172.21.15.131:0/4041398959 conn(0x7fba9c069840 msgr2=0x7fba9c14c190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:07.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.441+0000 7fbaa202b640 1 -- 172.21.15.131:0/4041398959 shutdown_connections 2023-09-12T14:55:07.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.441+0000 7fbaa202b640 1 -- 172.21.15.131:0/4041398959 wait complete. 2023-09-12T14:55:07.488 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T14:54:45.654004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T14:54:55.689983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T14:54:57.029447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-12T14:55:07.488 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T14:55:07.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.601+0000 7f3ab04fb640 1 -- 172.21.15.131:0/919694481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a90003f00 msgr2=0x7f3a900047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:07.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.601+0000 7f3ab04fb640 1 --2- 172.21.15.131:0/919694481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a90003f00 0x7f3a900047d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009b60 tx=0x7f3a9c01d1f0 comp rx=0 tx=0).stop 2023-09-12T14:55:07.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.601+0000 7f3aaea98640 1 -- 172.21.15.131:0/919694481 reap_dead start 2023-09-12T14:55:07.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.601+0000 7f3ab04fb640 1 -- 172.21.15.131:0/919694481 shutdown_connections 2023-09-12T14:55:07.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.601+0000 7f3ab04fb640 1 -- 172.21.15.131:0/919694481 >> 172.21.15.131:0/919694481 conn(0x7f3aa806a520 msgr2=0x7f3aa806a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:07.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.601+0000 7f3ab04fb640 1 -- 172.21.15.131:0/919694481 shutdown_connections 2023-09-12T14:55:07.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.601+0000 7f3ab04fb640 1 -- 172.21.15.131:0/919694481 wait complete. 2023-09-12T14:55:07.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.601+0000 7f3ab04fb640 1 Processor -- start 2023-09-12T14:55:07.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.601+0000 7f3ab04fb640 1 -- start start 2023-09-12T14:55:07.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.601+0000 7f3ab04fb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3aa80c9b50 0x7f3aa814f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:07.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.601+0000 7f3ab04fb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3aa814f9f0 0x7f3aa81531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:07.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.601+0000 7f3ab04fb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3aa8153730 0x7f3aa814cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:07.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.601+0000 7f3ab04fb640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa8153bc0 con 0x7f3aa80c9b50 2023-09-12T14:55:07.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.601+0000 7f3ab04fb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa80c9f30 con 0x7f3aa814f9f0 2023-09-12T14:55:07.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.601+0000 7f3ab04fb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa814fe70 con 0x7f3aa8153730 2023-09-12T14:55:07.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.601+0000 7f3aae297640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3aa80c9b50 0x7f3aa814f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:07.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.605+0000 7f3aae297640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3aa80c9b50 0x7f3aa814f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39310/0 (socket says 172.21.15.131:39310) 2023-09-12T14:55:07.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.605+0000 7f3aae297640 1 -- 172.21.15.131:0/4040261878 learned_addr learned my addr 172.21.15.131:0/4040261878 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:07.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.605+0000 7f3aada96640 1 --2- 172.21.15.131:0/4040261878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3aa814f9f0 0x7f3aa81531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:07.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.605+0000 7f3aaea98640 1 --2- 172.21.15.131:0/4040261878 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3aa8153730 0x7f3aa814cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:07.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.605+0000 7f3aae297640 1 -- 172.21.15.131:0/4040261878 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3aa8153730 msgr2=0x7f3aa814cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:07.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.605+0000 7f3aae297640 1 --2- 172.21.15.131:0/4040261878 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3aa8153730 0x7f3aa814cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:07.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.605+0000 7f3aae297640 1 -- 172.21.15.131:0/4040261878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3aa814f9f0 msgr2=0x7f3aa81531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:07.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.605+0000 7f3aae297640 1 --2- 172.21.15.131:0/4040261878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3aa814f9f0 0x7f3aa81531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:07.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.605+0000 7f3aae297640 1 -- 172.21.15.131:0/4040261878 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa80ce390 con 0x7f3aa80c9b50 2023-09-12T14:55:07.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.605+0000 7f3aae297640 1 --2- 172.21.15.131:0/4040261878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3aa80c9b50 0x7f3aa814f4b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0010960 tx=0x7f3aa0010e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:07.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.605+0000 7f3aa6ffd640 1 -- 172.21.15.131:0/4040261878 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0009730 con 0x7f3aa80c9b50 2023-09-12T14:55:07.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.605+0000 7f3aa6ffd640 1 -- 172.21.15.131:0/4040261878 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3aa000eda0 con 0x7f3aa80c9b50 2023-09-12T14:55:07.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.605+0000 7f3ab04fb640 1 -- 172.21.15.131:0/4040261878 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa8150150 con 0x7f3aa80c9b50 2023-09-12T14:55:07.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.605+0000 7f3aa6ffd640 1 -- 172.21.15.131:0/4040261878 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0005280 con 0x7f3aa80c9b50 2023-09-12T14:55:07.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.605+0000 7f3ab04fb640 1 -- 172.21.15.131:0/4040261878 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa81ef520 con 0x7f3aa80c9b50 2023-09-12T14:55:07.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.605+0000 7f3ab04fb640 1 -- 172.21.15.131:0/4040261878 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa806d720 con 0x7f3aa80c9b50 2023-09-12T14:55:07.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.605+0000 7f3aa6ffd640 1 -- 172.21.15.131:0/4040261878 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f3aa00053e0 con 0x7f3aa80c9b50 2023-09-12T14:55:07.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.605+0000 7f3aa6ffd640 1 --2- 172.21.15.131:0/4040261878 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3a94065b50 0x7f3a94068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:07.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.605+0000 7f3aada96640 1 --2- 172.21.15.131:0/4040261878 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3a94065b50 0x7f3a94068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:07.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.605+0000 7f3aa6ffd640 1 -- 172.21.15.131:0/4040261878 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3aa0016070 con 0x7f3aa80c9b50 2023-09-12T14:55:07.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.605+0000 7f3aada96640 1 --2- 172.21.15.131:0/4040261878 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3a94065b50 0x7f3a94068010 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0ae000 tx=0x7f3a9c019ff0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:07.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.609+0000 7f3aa6ffd640 1 -- 172.21.15.131:0/4040261878 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f3aa0051b20 con 0x7f3aa80c9b50 2023-09-12T14:55:07.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.749+0000 7f3ab04fb640 1 -- 172.21.15.131:0/4040261878 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3aa80cd0e0 con 0x7f3aa80c9b50 2023-09-12T14:55:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.753+0000 7f3aa6ffd640 1 -- 172.21.15.131:0/4040261878 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3aa0055140 con 0x7f3aa80c9b50 2023-09-12T14:55:07.754 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-09-12T14:55:07.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.753+0000 7f3ab04fb640 1 -- 172.21.15.131:0/4040261878 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3a94065b50 msgr2=0x7f3a94068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:07.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.753+0000 7f3ab04fb640 1 --2- 172.21.15.131:0/4040261878 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3a94065b50 0x7f3a94068010 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0ae000 tx=0x7f3a9c019ff0 comp rx=0 tx=0).stop 2023-09-12T14:55:07.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.753+0000 7f3ab04fb640 1 -- 172.21.15.131:0/4040261878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3aa80c9b50 msgr2=0x7f3aa814f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:07.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.753+0000 7f3ab04fb640 1 --2- 172.21.15.131:0/4040261878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3aa80c9b50 0x7f3aa814f4b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0010960 tx=0x7f3aa0010e20 comp rx=0 tx=0).stop 2023-09-12T14:55:07.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.753+0000 7f3ab04fb640 1 -- 172.21.15.131:0/4040261878 shutdown_connections 2023-09-12T14:55:07.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.753+0000 7f3ab04fb640 1 --2- 172.21.15.131:0/4040261878 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3a94065b50 0x7f3a94068010 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:07.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.753+0000 7f3ab04fb640 1 --2- 172.21.15.131:0/4040261878 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3aa8153730 0x7f3aa814cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:07.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.753+0000 7f3ab04fb640 1 --2- 172.21.15.131:0/4040261878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3aa814f9f0 0x7f3aa81531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:07.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.753+0000 7f3ab04fb640 1 --2- 172.21.15.131:0/4040261878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3aa80c9b50 0x7f3aa814f4b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:07.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.753+0000 7f3ab04fb640 1 -- 172.21.15.131:0/4040261878 >> 172.21.15.131:0/4040261878 conn(0x7f3aa806a520 msgr2=0x7f3aa8158030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:07.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.753+0000 7f3ab04fb640 1 -- 172.21.15.131:0/4040261878 shutdown_connections 2023-09-12T14:55:07.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.753+0000 7f3ab04fb640 1 -- 172.21.15.131:0/4040261878 wait complete. 2023-09-12T14:55:07.771 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T14:55:07.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.893+0000 7f31779ff640 1 -- 172.21.15.131:0/1726068279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f315c0039d0 msgr2=0x7f315c003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:07.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.893+0000 7f31779ff640 1 --2- 172.21.15.131:0/1726068279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f315c0039d0 0x7f315c003df0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f316c009b10 tx=0x7f316c01d1e0 comp rx=0 tx=0).stop 2023-09-12T14:55:07.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.893+0000 7f31779ff640 1 -- 172.21.15.131:0/1726068279 shutdown_connections 2023-09-12T14:55:07.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.893+0000 7f31779ff640 1 --2- 172.21.15.131:0/1726068279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f315c0039d0 0x7f315c003df0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:07.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.893+0000 7f31779ff640 1 -- 172.21.15.131:0/1726068279 >> 172.21.15.131:0/1726068279 conn(0x7f31700ceae0 msgr2=0x7f31700ceef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:07.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.893+0000 7f31779ff640 1 -- 172.21.15.131:0/1726068279 shutdown_connections 2023-09-12T14:55:07.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.893+0000 7f31779ff640 1 -- 172.21.15.131:0/1726068279 wait complete. 2023-09-12T14:55:07.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.893+0000 7f31779ff640 1 Processor -- start 2023-09-12T14:55:07.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.893+0000 7f31779ff640 1 -- start start 2023-09-12T14:55:07.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.893+0000 7f31779ff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f315c0039d0 0x7f31701466e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:07.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.893+0000 7f31779ff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31700b1700 0x7f31700b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:07.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.893+0000 7f31779ff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f317006a350 0x7f317006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:07.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.893+0000 7f31779ff640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31701491a0 con 0x7f317006a350 2023-09-12T14:55:07.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.893+0000 7f31779ff640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31700b20a0 con 0x7f31700b1700 2023-09-12T14:55:07.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.893+0000 7f31779ff640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f317006c070 con 0x7f315c0039d0 2023-09-12T14:55:07.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.893+0000 7f3175f9c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f317006a350 0x7f317006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:07.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.893+0000 7f3175f9c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f317006a350 0x7f317006a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39324/0 (socket says 172.21.15.131:39324) 2023-09-12T14:55:07.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.893+0000 7f3175f9c640 1 -- 172.21.15.131:0/3233996547 learned_addr learned my addr 172.21.15.131:0/3233996547 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:07.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.893+0000 7f3174f9a640 1 --2- 172.21.15.131:0/3233996547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31700b1700 0x7f31700b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:07.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.893+0000 7f3175f9c640 1 -- 172.21.15.131:0/3233996547 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f315c0039d0 msgr2=0x7f31701466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:07.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.893+0000 7f3175f9c640 1 --2- 172.21.15.131:0/3233996547 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f315c0039d0 0x7f31701466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:07.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.893+0000 7f3175f9c640 1 -- 172.21.15.131:0/3233996547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31700b1700 msgr2=0x7f31700b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:07.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.893+0000 7f3175f9c640 1 --2- 172.21.15.131:0/3233996547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31700b1700 0x7f31700b1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:07.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.893+0000 7f3175f9c640 1 -- 172.21.15.131:0/3233996547 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317006ad10 con 0x7f317006a350 2023-09-12T14:55:07.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.897+0000 7f3175f9c640 1 --2- 172.21.15.131:0/3233996547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f317006a350 0x7f317006a7d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f317015c920 tx=0x7f316800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:07.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.897+0000 7f3165ffb640 1 -- 172.21.15.131:0/3233996547 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3168006d60 con 0x7f317006a350 2023-09-12T14:55:07.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.897+0000 7f3165ffb640 1 -- 172.21.15.131:0/3233996547 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f316800ced0 con 0x7f317006a350 2023-09-12T14:55:07.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.897+0000 7f31779ff640 1 -- 172.21.15.131:0/3233996547 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f317006afd0 con 0x7f317006a350 2023-09-12T14:55:07.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.897+0000 7f3165ffb640 1 -- 172.21.15.131:0/3233996547 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3168003be0 con 0x7f317006a350 2023-09-12T14:55:07.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.897+0000 7f31779ff640 1 -- 172.21.15.131:0/3233996547 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f317006b590 con 0x7f317006a350 2023-09-12T14:55:07.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.897+0000 7f31779ff640 1 -- 172.21.15.131:0/3233996547 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3170073580 con 0x7f317006a350 2023-09-12T14:55:07.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.897+0000 7f3165ffb640 1 -- 172.21.15.131:0/3233996547 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f3168003d40 con 0x7f317006a350 2023-09-12T14:55:07.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.897+0000 7f3165ffb640 1 --2- 172.21.15.131:0/3233996547 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3154065b20 0x7f3154067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:07.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.897+0000 7f3165ffb640 1 -- 172.21.15.131:0/3233996547 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3168082de0 con 0x7f317006a350 2023-09-12T14:55:07.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.897+0000 7f317579b640 1 --2- 172.21.15.131:0/3233996547 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3154065b20 0x7f3154067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:07.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.897+0000 7f317579b640 1 --2- 172.21.15.131:0/3233996547 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3154065b20 0x7f3154067fe0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f316c0ad000 tx=0x7f315c005800 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:07.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:07.901+0000 7f3165ffb640 1 -- 172.21.15.131:0/3233996547 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f31680507c0 con 0x7f317006a350 2023-09-12T14:55:08.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.041+0000 7f31779ff640 1 -- 172.21.15.131:0/3233996547 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f31700c5870 con 0x7f317006a350 2023-09-12T14:55:08.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.041+0000 7f3165ffb640 1 -- 172.21.15.131:0/3233996547 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3168053de0 con 0x7f317006a350 2023-09-12T14:55:08.045 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-09-12T14:55:08.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.045+0000 7f31779ff640 1 -- 172.21.15.131:0/3233996547 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3154065b20 msgr2=0x7f3154067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:08.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.045+0000 7f31779ff640 1 --2- 172.21.15.131:0/3233996547 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3154065b20 0x7f3154067fe0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f316c0ad000 tx=0x7f315c005800 comp rx=0 tx=0).stop 2023-09-12T14:55:08.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.045+0000 7f31779ff640 1 -- 172.21.15.131:0/3233996547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f317006a350 msgr2=0x7f317006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:08.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.045+0000 7f31779ff640 1 --2- 172.21.15.131:0/3233996547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f317006a350 0x7f317006a7d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f317015c920 tx=0x7f316800cb90 comp rx=0 tx=0).stop 2023-09-12T14:55:08.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.045+0000 7f31779ff640 1 -- 172.21.15.131:0/3233996547 shutdown_connections 2023-09-12T14:55:08.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.045+0000 7f31779ff640 1 --2- 172.21.15.131:0/3233996547 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3154065b20 0x7f3154067fe0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.045+0000 7f31779ff640 1 --2- 172.21.15.131:0/3233996547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f317006a350 0x7f317006a7d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.045+0000 7f31779ff640 1 --2- 172.21.15.131:0/3233996547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31700b1700 0x7f31700b1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.045+0000 7f31779ff640 1 --2- 172.21.15.131:0/3233996547 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f315c0039d0 0x7f31701466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.045+0000 7f31779ff640 1 -- 172.21.15.131:0/3233996547 >> 172.21.15.131:0/3233996547 conn(0x7f31700ceae0 msgr2=0x7f31700ab740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:08.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.045+0000 7f31779ff640 1 -- 172.21.15.131:0/3233996547 shutdown_connections 2023-09-12T14:55:08.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.045+0000 7f31779ff640 1 -- 172.21.15.131:0/3233996547 wait complete. 2023-09-12T14:55:08.062 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T14:55:08.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f66319e7640 1 -- 172.21.15.131:0/2625593979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f662c06b050 msgr2=0x7f662c06b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:08.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f66319e7640 1 --2- 172.21.15.131:0/2625593979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f662c06b050 0x7f662c06b500 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6624009c80 tx=0x7f662401cd10 comp rx=0 tx=0).stop 2023-09-12T14:55:08.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f66319e7640 1 -- 172.21.15.131:0/2625593979 shutdown_connections 2023-09-12T14:55:08.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f66319e7640 1 --2- 172.21.15.131:0/2625593979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f662c06b050 0x7f662c06b500 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f66319e7640 1 --2- 172.21.15.131:0/2625593979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f662c158180 0x7f662c0cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f66319e7640 1 -- 172.21.15.131:0/2625593979 >> 172.21.15.131:0/2625593979 conn(0x7f662c069840 msgr2=0x7f662c0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:08.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f66319e7640 1 -- 172.21.15.131:0/2625593979 shutdown_connections 2023-09-12T14:55:08.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f66319e7640 1 -- 172.21.15.131:0/2625593979 wait complete. 2023-09-12T14:55:08.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f66319e7640 1 Processor -- start 2023-09-12T14:55:08.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f66319e7640 1 -- start start 2023-09-12T14:55:08.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f66319e7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f662c06b050 0x7f662c14f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:08.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f66319e7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f662c0ab730 0x7f662c142b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:08.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f66319e7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f662c158180 0x7f662c14df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:08.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f66319e7640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662c14fec0 con 0x7f662c158180 2023-09-12T14:55:08.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f66319e7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662c14e8e0 con 0x7f662c0ab730 2023-09-12T14:55:08.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f66319e7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f662c14ebc0 con 0x7f662c06b050 2023-09-12T14:55:08.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f662b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f662c158180 0x7f662c14df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:08.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f662b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f662c158180 0x7f662c14df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39342/0 (socket says 172.21.15.131:39342) 2023-09-12T14:55:08.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f662b7fe640 1 -- 172.21.15.131:0/2456669249 learned_addr learned my addr 172.21.15.131:0/2456669249 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:08.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f662a7fc640 1 --2- 172.21.15.131:0/2456669249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f662c0ab730 0x7f662c142b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:08.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f662b7fe640 1 -- 172.21.15.131:0/2456669249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f662c06b050 msgr2=0x7f662c14f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:08.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f662affd640 1 --2- 172.21.15.131:0/2456669249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f662c06b050 0x7f662c14f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:08.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f662b7fe640 1 --2- 172.21.15.131:0/2456669249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f662c06b050 0x7f662c14f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f662b7fe640 1 -- 172.21.15.131:0/2456669249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f662c0ab730 msgr2=0x7f662c142b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:08.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f662b7fe640 1 --2- 172.21.15.131:0/2456669249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f662c0ab730 0x7f662c142b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f662b7fe640 1 -- 172.21.15.131:0/2456669249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66240098e0 con 0x7f662c158180 2023-09-12T14:55:08.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f662a7fc640 1 --2- 172.21.15.131:0/2456669249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f662c0ab730 0x7f662c142b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:08.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.177+0000 7f662b7fe640 1 --2- 172.21.15.131:0/2456669249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f662c158180 0x7f662c14df70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f662401b3c0 tx=0x7f6624003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:08.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.181+0000 7f6613fff640 1 -- 172.21.15.131:0/2456669249 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66240bc070 con 0x7f662c158180 2023-09-12T14:55:08.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.181+0000 7f6613fff640 1 -- 172.21.15.131:0/2456669249 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f66240092c0 con 0x7f662c158180 2023-09-12T14:55:08.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.181+0000 7f66319e7640 1 -- 172.21.15.131:0/2456669249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f662c14eea0 con 0x7f662c158180 2023-09-12T14:55:08.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.181+0000 7f6613fff640 1 -- 172.21.15.131:0/2456669249 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66240adaf0 con 0x7f662c158180 2023-09-12T14:55:08.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.181+0000 7f66319e7640 1 -- 172.21.15.131:0/2456669249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f662c14f460 con 0x7f662c158180 2023-09-12T14:55:08.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.181+0000 7f66319e7640 1 -- 172.21.15.131:0/2456669249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f662c14f650 con 0x7f662c158180 2023-09-12T14:55:08.189 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.181+0000 7f6613fff640 1 -- 172.21.15.131:0/2456669249 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f66240b4040 con 0x7f662c158180 2023-09-12T14:55:08.189 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.181+0000 7f6613fff640 1 --2- 172.21.15.131:0/2456669249 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6618065b20 0x7f6618067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:08.189 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.181+0000 7f6613fff640 1 -- 172.21.15.131:0/2456669249 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f662411e070 con 0x7f662c158180 2023-09-12T14:55:08.189 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.181+0000 7f662affd640 1 --2- 172.21.15.131:0/2456669249 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6618065b20 0x7f6618067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:08.189 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.181+0000 7f662affd640 1 --2- 172.21.15.131:0/2456669249 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6618065b20 0x7f6618067fe0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f661c003d60 tx=0x7f661c009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:08.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.185+0000 7f6613fff640 1 -- 172.21.15.131:0/2456669249 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f66240eb9c0 con 0x7f662c158180 2023-09-12T14:55:08.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.325+0000 7f66319e7640 1 -- 172.21.15.131:0/2456669249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f662c073580 con 0x7f662c158180 2023-09-12T14:55:08.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.325+0000 7f6613fff640 1 -- 172.21.15.131:0/2456669249 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f66240eefe0 con 0x7f662c158180 2023-09-12T14:55:08.327 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-09-12T14:55:08.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.325+0000 7f66319e7640 1 -- 172.21.15.131:0/2456669249 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6618065b20 msgr2=0x7f6618067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:08.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.325+0000 7f66319e7640 1 --2- 172.21.15.131:0/2456669249 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6618065b20 0x7f6618067fe0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f661c003d60 tx=0x7f661c009040 comp rx=0 tx=0).stop 2023-09-12T14:55:08.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.325+0000 7f66319e7640 1 -- 172.21.15.131:0/2456669249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f662c158180 msgr2=0x7f662c14df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:08.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.325+0000 7f66319e7640 1 --2- 172.21.15.131:0/2456669249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f662c158180 0x7f662c14df70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f662401b3c0 tx=0x7f6624003cf0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.325+0000 7f66319e7640 1 -- 172.21.15.131:0/2456669249 shutdown_connections 2023-09-12T14:55:08.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.325+0000 7f66319e7640 1 --2- 172.21.15.131:0/2456669249 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6618065b20 0x7f6618067fe0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.325+0000 7f66319e7640 1 --2- 172.21.15.131:0/2456669249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f662c158180 0x7f662c14df70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.325+0000 7f66319e7640 1 --2- 172.21.15.131:0/2456669249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f662c0ab730 0x7f662c142b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.325+0000 7f66319e7640 1 --2- 172.21.15.131:0/2456669249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f662c06b050 0x7f662c14f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.325+0000 7f66319e7640 1 -- 172.21.15.131:0/2456669249 >> 172.21.15.131:0/2456669249 conn(0x7f662c069840 msgr2=0x7f662c0abb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:08.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.325+0000 7f66319e7640 1 -- 172.21.15.131:0/2456669249 shutdown_connections 2023-09-12T14:55:08.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.329+0000 7f66319e7640 1 -- 172.21.15.131:0/2456669249 wait complete. 2023-09-12T14:55:08.345 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T14:55:08.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.465+0000 7f5a564e6640 1 -- 172.21.15.131:0/2744506523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a3c005190 msgr2=0x7f5a3c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:08.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.465+0000 7f5a564e6640 1 --2- 172.21.15.131:0/2744506523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a3c005190 0x7f5a3c00b990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5a48009eb0 tx=0x7f5a4801d5d0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.465+0000 7f5a564e6640 1 -- 172.21.15.131:0/2744506523 shutdown_connections 2023-09-12T14:55:08.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.465+0000 7f5a564e6640 1 --2- 172.21.15.131:0/2744506523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a3c005190 0x7f5a3c00b990 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.465+0000 7f5a564e6640 1 -- 172.21.15.131:0/2744506523 >> 172.21.15.131:0/2744506523 conn(0x7f5a50069840 msgr2=0x7f5a500cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:08.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.465+0000 7f5a564e6640 1 -- 172.21.15.131:0/2744506523 shutdown_connections 2023-09-12T14:55:08.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.465+0000 7f5a564e6640 1 -- 172.21.15.131:0/2744506523 wait complete. 2023-09-12T14:55:08.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.465+0000 7f5a564e6640 1 Processor -- start 2023-09-12T14:55:08.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.465+0000 7f5a564e6640 1 -- start start 2023-09-12T14:55:08.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.465+0000 7f5a564e6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a3c005190 0x7f5a50151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:08.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.465+0000 7f5a564e6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a501f0b60 0x7f5a501452f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:08.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.465+0000 7f5a564e6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a501f4820 0x7f5a501f4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:08.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.465+0000 7f5a564e6640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a501f1bc0 con 0x7f5a501f0b60 2023-09-12T14:55:08.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.465+0000 7f5a564e6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a50151c90 con 0x7f5a501f4820 2023-09-12T14:55:08.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.465+0000 7f5a564e6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a501f1320 con 0x7f5a3c005190 2023-09-12T14:55:08.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.465+0000 7f5a4f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a501f0b60 0x7f5a501452f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:08.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.469+0000 7f5a4f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a501f0b60 0x7f5a501452f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41330/0 (socket says 172.21.15.131:41330) 2023-09-12T14:55:08.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.469+0000 7f5a4f7fe640 1 -- 172.21.15.131:0/4293487383 learned_addr learned my addr 172.21.15.131:0/4293487383 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:08.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.469+0000 7f5a54a83640 1 --2- 172.21.15.131:0/4293487383 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a501f4820 0x7f5a501f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:08.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.469+0000 7f5a4ffff640 1 --2- 172.21.15.131:0/4293487383 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a3c005190 0x7f5a50151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:08.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.469+0000 7f5a4f7fe640 1 -- 172.21.15.131:0/4293487383 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a3c005190 msgr2=0x7f5a50151420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:08.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.469+0000 7f5a4f7fe640 1 --2- 172.21.15.131:0/4293487383 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a3c005190 0x7f5a50151420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.469+0000 7f5a4f7fe640 1 -- 172.21.15.131:0/4293487383 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a501f4820 msgr2=0x7f5a501f4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:08.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.469+0000 7f5a4f7fe640 1 --2- 172.21.15.131:0/4293487383 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a501f4820 0x7f5a501f4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.469+0000 7f5a4f7fe640 1 -- 172.21.15.131:0/4293487383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a501f1600 con 0x7f5a501f0b60 2023-09-12T14:55:08.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.469+0000 7f5a4f7fe640 1 --2- 172.21.15.131:0/4293487383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a501f0b60 0x7f5a501452f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5a38006680 tx=0x7f5a380039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:08.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.469+0000 7f5a4cff9640 1 -- 172.21.15.131:0/4293487383 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a38011900 con 0x7f5a501f0b60 2023-09-12T14:55:08.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.469+0000 7f5a4cff9640 1 -- 172.21.15.131:0/4293487383 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5a38011a60 con 0x7f5a501f0b60 2023-09-12T14:55:08.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.469+0000 7f5a4cff9640 1 -- 172.21.15.131:0/4293487383 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a38011e70 con 0x7f5a501f0b60 2023-09-12T14:55:08.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.469+0000 7f5a564e6640 1 -- 172.21.15.131:0/4293487383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a501f18e0 con 0x7f5a501f0b60 2023-09-12T14:55:08.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.469+0000 7f5a564e6640 1 -- 172.21.15.131:0/4293487383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a50145830 con 0x7f5a501f0b60 2023-09-12T14:55:08.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.469+0000 7f5a564e6640 1 -- 172.21.15.131:0/4293487383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a501f4160 con 0x7f5a501f0b60 2023-09-12T14:55:08.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.469+0000 7f5a4cff9640 1 -- 172.21.15.131:0/4293487383 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f5a380103c0 con 0x7f5a501f0b60 2023-09-12T14:55:08.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.469+0000 7f5a4cff9640 1 --2- 172.21.15.131:0/4293487383 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5a30065b50 0x7f5a30068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:08.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.469+0000 7f5a4cff9640 1 -- 172.21.15.131:0/4293487383 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a380841c0 con 0x7f5a501f0b60 2023-09-12T14:55:08.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.469+0000 7f5a4ffff640 1 --2- 172.21.15.131:0/4293487383 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5a30065b50 0x7f5a30068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:08.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.469+0000 7f5a4ffff640 1 --2- 172.21.15.131:0/4293487383 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5a30065b50 0x7f5a30068010 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5a50151f50 tx=0x7f5a40006bf0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:08.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.473+0000 7f5a4cff9640 1 -- 172.21.15.131:0/4293487383 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f5a501f4160 con 0x7f5a501f0b60 2023-09-12T14:55:08.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.661+0000 7f5a564e6640 1 -- 172.21.15.131:0/4293487383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a500cc090 con 0x7f5a501f0b60 2023-09-12T14:55:08.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.661+0000 7f5a4cff9640 1 -- 172.21.15.131:0/4293487383 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f5a500cc090 con 0x7f5a501f0b60 2023-09-12T14:55:08.665 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:08.665 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4344,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/491327908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":491327908},{"type":"v1","addr":"172.21.15.131:6837","nonce":491327908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3470212105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3470212105},{"type":"v1","addr":"172.21.15.187:6833","nonce":3470212105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/91478708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":91478708},{"type":"v1","addr":"172.21.15.131:6836","nonce":91478708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-12T14:54:58.030444+0000","modified":"2023-09-12T14:55:00.236338+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4396":{"gid":4396,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/290727181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":290727181},{"type":"v1","addr":"172.21.15.187:6835","nonce":290727181}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-12T14:55:08.666 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 6 2023-09-12T14:55:08.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.665+0000 7f5a564e6640 1 -- 172.21.15.131:0/4293487383 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5a30065b50 msgr2=0x7f5a30068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:08.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.665+0000 7f5a564e6640 1 --2- 172.21.15.131:0/4293487383 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5a30065b50 0x7f5a30068010 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5a50151f50 tx=0x7f5a40006bf0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.665+0000 7f5a564e6640 1 -- 172.21.15.131:0/4293487383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a501f0b60 msgr2=0x7f5a501452f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:08.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.665+0000 7f5a564e6640 1 --2- 172.21.15.131:0/4293487383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a501f0b60 0x7f5a501452f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5a38006680 tx=0x7f5a380039d0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.665+0000 7f5a564e6640 1 -- 172.21.15.131:0/4293487383 shutdown_connections 2023-09-12T14:55:08.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.665+0000 7f5a564e6640 1 --2- 172.21.15.131:0/4293487383 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5a30065b50 0x7f5a30068010 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.665+0000 7f5a564e6640 1 --2- 172.21.15.131:0/4293487383 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a501f4820 0x7f5a501f4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.665+0000 7f5a564e6640 1 --2- 172.21.15.131:0/4293487383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a501f0b60 0x7f5a501452f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.665+0000 7f5a564e6640 1 --2- 172.21.15.131:0/4293487383 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a3c005190 0x7f5a50151420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.665+0000 7f5a564e6640 1 -- 172.21.15.131:0/4293487383 >> 172.21.15.131:0/4293487383 conn(0x7f5a50069840 msgr2=0x7f5a5006eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:08.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.665+0000 7f5a564e6640 1 -- 172.21.15.131:0/4293487383 shutdown_connections 2023-09-12T14:55:08.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.665+0000 7f5a564e6640 1 -- 172.21.15.131:0/4293487383 wait complete. 2023-09-12T14:55:08.684 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:55:08.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03aa421640 1 -- 172.21.15.131:0/1804223166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f038c005e20 msgr2=0x7f038c0062a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:08.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03aa421640 1 --2- 172.21.15.131:0/1804223166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f038c005e20 0x7f038c0062a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f039c001d70 tx=0x7f039c01caa0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03a89be640 1 -- 172.21.15.131:0/1804223166 reap_dead start 2023-09-12T14:55:08.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03aa421640 1 -- 172.21.15.131:0/1804223166 shutdown_connections 2023-09-12T14:55:08.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03aa421640 1 -- 172.21.15.131:0/1804223166 >> 172.21.15.131:0/1804223166 conn(0x7f03a4069840 msgr2=0x7f03a40cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:08.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03aa421640 1 -- 172.21.15.131:0/1804223166 shutdown_connections 2023-09-12T14:55:08.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03aa421640 1 -- 172.21.15.131:0/1804223166 wait complete. 2023-09-12T14:55:08.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03aa421640 1 Processor -- start 2023-09-12T14:55:08.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03aa421640 1 -- start start 2023-09-12T14:55:08.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03aa421640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f038c005e20 0x7f03a41484b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:08.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03aa421640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03a4159930 0x7f03a41583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:08.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03aa421640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03a41446e0 0x7f03a40ac620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:08.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03aa421640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a414b900 con 0x7f038c005e20 2023-09-12T14:55:08.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03aa421640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a414ac80 con 0x7f03a41446e0 2023-09-12T14:55:08.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03aa421640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a414af60 con 0x7f03a4159930 2023-09-12T14:55:08.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03a3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f038c005e20 0x7f03a41484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:08.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03a3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f038c005e20 0x7f03a41484b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41354/0 (socket says 172.21.15.131:41354) 2023-09-12T14:55:08.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03a3fff640 1 -- 172.21.15.131:0/265302256 learned_addr learned my addr 172.21.15.131:0/265302256 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:08.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03a89be640 1 --2- 172.21.15.131:0/265302256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03a41446e0 0x7f03a40ac620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:08.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03a37fe640 1 --2- 172.21.15.131:0/265302256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03a4159930 0x7f03a41583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:08.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03a3fff640 1 -- 172.21.15.131:0/265302256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03a4159930 msgr2=0x7f03a41583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:08.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03a3fff640 1 --2- 172.21.15.131:0/265302256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03a4159930 0x7f03a41583b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03a3fff640 1 -- 172.21.15.131:0/265302256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03a41446e0 msgr2=0x7f03a40ac620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:08.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03a3fff640 1 --2- 172.21.15.131:0/265302256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03a41446e0 0x7f03a40ac620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03a3fff640 1 -- 172.21.15.131:0/265302256 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a414b240 con 0x7f038c005e20 2023-09-12T14:55:08.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03a37fe640 1 --2- 172.21.15.131:0/265302256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03a4159930 0x7f03a41583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:08.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03a3fff640 1 --2- 172.21.15.131:0/265302256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f038c005e20 0x7f03a41484b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f039400f9d0 tx=0x7f039400fe90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:08.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03a0ff9640 1 -- 172.21.15.131:0/265302256 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039400bcb0 con 0x7f038c005e20 2023-09-12T14:55:08.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03a0ff9640 1 -- 172.21.15.131:0/265302256 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0394011440 con 0x7f038c005e20 2023-09-12T14:55:08.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.805+0000 7f03aa421640 1 -- 172.21.15.131:0/265302256 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a414b520 con 0x7f038c005e20 2023-09-12T14:55:08.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.809+0000 7f03a0ff9640 1 -- 172.21.15.131:0/265302256 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0394011810 con 0x7f038c005e20 2023-09-12T14:55:08.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.809+0000 7f03aa421640 1 -- 172.21.15.131:0/265302256 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a4144b60 con 0x7f038c005e20 2023-09-12T14:55:08.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.809+0000 7f03aa421640 1 -- 172.21.15.131:0/265302256 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a4073580 con 0x7f038c005e20 2023-09-12T14:55:08.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.809+0000 7f03a0ff9640 1 -- 172.21.15.131:0/265302256 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f0394011970 con 0x7f038c005e20 2023-09-12T14:55:08.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.809+0000 7f03a0ff9640 1 --2- 172.21.15.131:0/265302256 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0390065b50 0x7f0390068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:08.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.809+0000 7f03a37fe640 1 --2- 172.21.15.131:0/265302256 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0390065b50 0x7f0390068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:08.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.809+0000 7f03a0ff9640 1 -- 172.21.15.131:0/265302256 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0394083f00 con 0x7f038c005e20 2023-09-12T14:55:08.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.809+0000 7f03a37fe640 1 --2- 172.21.15.131:0/265302256 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0390065b50 0x7f0390068010 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f03a41591f0 tx=0x7f0398001510 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:08.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.813+0000 7f03a0ff9640 1 -- 172.21.15.131:0/265302256 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f03940518e0 con 0x7f038c005e20 2023-09-12T14:55:08.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.957+0000 7f03aa421640 1 -- 172.21.15.131:0/265302256 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03a406c0a0 con 0x7f038c005e20 2023-09-12T14:55:08.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.957+0000 7f03a0ff9640 1 -- 172.21.15.131:0/265302256 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f0394054f00 con 0x7f038c005e20 2023-09-12T14:55:08.960 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:08.960 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:54:59.694778+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:54:55.689983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T14:54:57.029447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:55:08.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.961+0000 7f03aa421640 1 -- 172.21.15.131:0/265302256 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0390065b50 msgr2=0x7f0390068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:08.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.961+0000 7f03aa421640 1 --2- 172.21.15.131:0/265302256 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0390065b50 0x7f0390068010 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f03a41591f0 tx=0x7f0398001510 comp rx=0 tx=0).stop 2023-09-12T14:55:08.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.961+0000 7f03aa421640 1 -- 172.21.15.131:0/265302256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f038c005e20 msgr2=0x7f03a41484b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:08.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.961+0000 7f03aa421640 1 --2- 172.21.15.131:0/265302256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f038c005e20 0x7f03a41484b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f039400f9d0 tx=0x7f039400fe90 comp rx=0 tx=0).stop 2023-09-12T14:55:08.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.961+0000 7f03aa421640 1 -- 172.21.15.131:0/265302256 shutdown_connections 2023-09-12T14:55:08.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.961+0000 7f03aa421640 1 --2- 172.21.15.131:0/265302256 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0390065b50 0x7f0390068010 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.961+0000 7f03aa421640 1 --2- 172.21.15.131:0/265302256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03a4159930 0x7f03a41583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.961+0000 7f03aa421640 1 --2- 172.21.15.131:0/265302256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03a41446e0 0x7f03a40ac620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.961+0000 7f03aa421640 1 --2- 172.21.15.131:0/265302256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f038c005e20 0x7f03a41484b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:08.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.961+0000 7f03aa421640 1 -- 172.21.15.131:0/265302256 >> 172.21.15.131:0/265302256 conn(0x7f03a4069840 msgr2=0x7f03a40adad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:08.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.961+0000 7f03aa421640 1 -- 172.21.15.131:0/265302256 shutdown_connections 2023-09-12T14:55:08.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:08.961+0000 7f03aa421640 1 -- 172.21.15.131:0/265302256 wait complete. 2023-09-12T14:55:08.978 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T14:54:45.654004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T14:54:55.689983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T14:54:57.029447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-12T14:55:08.978 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T14:55:09.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca6fa2640 1 -- 172.21.15.131:0/896874305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ca00d1680 msgr2=0x7f9ca015e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:09.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca6fa2640 1 --2- 172.21.15.131:0/896874305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ca00d1680 0x7f9ca015e820 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9ca008b660 tx=0x7f9c9001c8a0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca6fa2640 1 -- 172.21.15.131:0/896874305 shutdown_connections 2023-09-12T14:55:09.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca6fa2640 1 --2- 172.21.15.131:0/896874305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ca00cf6b0 0x7f9ca00aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca6fa2640 1 --2- 172.21.15.131:0/896874305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ca00d1680 0x7f9ca015e820 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca6fa2640 1 --2- 172.21.15.131:0/896874305 >> v2:172.21.15.187:3300/0 conn(0x7f9ca00cf2d0 0x7f9ca015e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca6fa2640 1 -- 172.21.15.131:0/896874305 >> 172.21.15.131:0/896874305 conn(0x7f9ca009a200 msgr2=0x7f9ca00d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:09.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca6fa2640 1 -- 172.21.15.131:0/896874305 shutdown_connections 2023-09-12T14:55:09.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca6fa2640 1 -- 172.21.15.131:0/896874305 wait complete. 2023-09-12T14:55:09.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca6fa2640 1 Processor -- start 2023-09-12T14:55:09.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca6fa2640 1 -- start start 2023-09-12T14:55:09.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca6fa2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ca00cf2d0 0x7f9ca014d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca6fa2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ca00cf6b0 0x7f9ca015c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca6fa2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ca00d1680 0x7f9ca015cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca6fa2640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca00b5290 con 0x7f9ca00cf2d0 2023-09-12T14:55:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca6fa2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca00b5570 con 0x7f9ca00d1680 2023-09-12T14:55:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca6fa2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca00b5850 con 0x7f9ca00cf6b0 2023-09-12T14:55:09.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca4d3e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ca00cf2d0 0x7f9ca014d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:09.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca4d3e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ca00cf2d0 0x7f9ca014d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41372/0 (socket says 172.21.15.131:41372) 2023-09-12T14:55:09.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca4d3e640 1 -- 172.21.15.131:0/691635650 learned_addr learned my addr 172.21.15.131:0/691635650 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:09.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9c97fff640 1 --2- 172.21.15.131:0/691635650 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ca00cf6b0 0x7f9ca015c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:09.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca553f640 1 --2- 172.21.15.131:0/691635650 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ca00d1680 0x7f9ca015cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca4d3e640 1 -- 172.21.15.131:0/691635650 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ca00cf6b0 msgr2=0x7f9ca015c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca4d3e640 1 --2- 172.21.15.131:0/691635650 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ca00cf6b0 0x7f9ca015c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca4d3e640 1 -- 172.21.15.131:0/691635650 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ca00d1680 msgr2=0x7f9ca015cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca4d3e640 1 --2- 172.21.15.131:0/691635650 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ca00d1680 0x7f9ca015cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca4d3e640 1 -- 172.21.15.131:0/691635650 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c90009560 con 0x7f9ca00cf2d0 2023-09-12T14:55:09.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca4d3e640 1 --2- 172.21.15.131:0/691635650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ca00cf2d0 0x7f9ca014d290 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c00d770 tx=0x7f9c9c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:09.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9c957fa640 1 -- 172.21.15.131:0/691635650 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c009c30 con 0x7f9ca00cf2d0 2023-09-12T14:55:09.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9c957fa640 1 -- 172.21.15.131:0/691635650 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9c9c009e80 con 0x7f9ca00cf2d0 2023-09-12T14:55:09.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca6fa2640 1 -- 172.21.15.131:0/691635650 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca00b5b30 con 0x7f9ca00cf2d0 2023-09-12T14:55:09.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9c957fa640 1 -- 172.21.15.131:0/691635650 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c0106a0 con 0x7f9ca00cf2d0 2023-09-12T14:55:09.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca6fa2640 1 -- 172.21.15.131:0/691635650 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca014d7d0 con 0x7f9ca00cf2d0 2023-09-12T14:55:09.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.093+0000 7f9ca6fa2640 1 -- 172.21.15.131:0/691635650 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca0073580 con 0x7f9ca00cf2d0 2023-09-12T14:55:09.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.097+0000 7f9c957fa640 1 -- 172.21.15.131:0/691635650 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f9c9c010800 con 0x7f9ca00cf2d0 2023-09-12T14:55:09.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.097+0000 7f9c957fa640 1 --2- 172.21.15.131:0/691635650 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9c8c065b20 0x7f9c8c067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:09.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.097+0000 7f9c97fff640 1 --2- 172.21.15.131:0/691635650 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9c8c065b20 0x7f9c8c067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:09.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.097+0000 7f9c957fa640 1 -- 172.21.15.131:0/691635650 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c9c083ca0 con 0x7f9ca00cf2d0 2023-09-12T14:55:09.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.097+0000 7f9c97fff640 1 --2- 172.21.15.131:0/691635650 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9c8c065b20 0x7f9c8c067fe0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9c900af000 tx=0x7f9c9001b0e0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:09.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.101+0000 7f9c957fa640 1 -- 172.21.15.131:0/691635650 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f9c9c055060 con 0x7f9ca00cf2d0 2023-09-12T14:55:09.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.249+0000 7f9ca6fa2640 1 -- 172.21.15.131:0/691635650 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ca006a030 con 0x7f9ca00cf2d0 2023-09-12T14:55:09.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.249+0000 7f9c957fa640 1 -- 172.21.15.131:0/691635650 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9c9c0170a0 con 0x7f9ca00cf2d0 2023-09-12T14:55:09.253 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-09-12T14:55:09.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.253+0000 7f9ca6fa2640 1 -- 172.21.15.131:0/691635650 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9c8c065b20 msgr2=0x7f9c8c067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:09.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.253+0000 7f9ca6fa2640 1 --2- 172.21.15.131:0/691635650 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9c8c065b20 0x7f9c8c067fe0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9c900af000 tx=0x7f9c9001b0e0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.253+0000 7f9ca6fa2640 1 -- 172.21.15.131:0/691635650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ca00cf2d0 msgr2=0x7f9ca014d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:09.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.253+0000 7f9ca6fa2640 1 --2- 172.21.15.131:0/691635650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ca00cf2d0 0x7f9ca014d290 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c00d770 tx=0x7f9c9c00dc30 comp rx=0 tx=0).stop 2023-09-12T14:55:09.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.253+0000 7f9ca6fa2640 1 -- 172.21.15.131:0/691635650 shutdown_connections 2023-09-12T14:55:09.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.253+0000 7f9ca6fa2640 1 --2- 172.21.15.131:0/691635650 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9c8c065b20 0x7f9c8c067fe0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.253+0000 7f9ca6fa2640 1 --2- 172.21.15.131:0/691635650 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ca00cf6b0 0x7f9ca015c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.253+0000 7f9ca6fa2640 1 --2- 172.21.15.131:0/691635650 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ca00d1680 0x7f9ca015cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.253+0000 7f9ca6fa2640 1 --2- 172.21.15.131:0/691635650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ca00cf2d0 0x7f9ca014d290 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.253+0000 7f9ca6fa2640 1 -- 172.21.15.131:0/691635650 >> 172.21.15.131:0/691635650 conn(0x7f9ca009a200 msgr2=0x7f9ca015e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:09.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.253+0000 7f9ca6fa2640 1 -- 172.21.15.131:0/691635650 shutdown_connections 2023-09-12T14:55:09.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.253+0000 7f9ca6fa2640 1 -- 172.21.15.131:0/691635650 wait complete. 2023-09-12T14:55:09.271 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T14:55:09.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc6a083d640 1 -- 172.21.15.131:0/3194775750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc69c15b140 msgr2=0x7fc69c0afd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:09.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc6a083d640 1 --2- 172.21.15.131:0/3194775750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc69c15b140 0x7fc69c0afd90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0d0790 tx=0x7fc69001c830 comp rx=0 tx=0).stop 2023-09-12T14:55:09.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc6a083d640 1 -- 172.21.15.131:0/3194775750 shutdown_connections 2023-09-12T14:55:09.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc6a083d640 1 --2- 172.21.15.131:0/3194775750 >> v2:172.21.15.187:3300/0 conn(0x7fc69c0b02d0 0x7fc69c0cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc6a083d640 1 --2- 172.21.15.131:0/3194775750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc69c15b140 0x7fc69c0afd90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc6a083d640 1 -- 172.21.15.131:0/3194775750 >> 172.21.15.131:0/3194775750 conn(0x7fc69c069bb0 msgr2=0x7fc69c069f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:09.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc6a083d640 1 -- 172.21.15.131:0/3194775750 shutdown_connections 2023-09-12T14:55:09.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc6a083d640 1 -- 172.21.15.131:0/3194775750 wait complete. 2023-09-12T14:55:09.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc6a083d640 1 Processor -- start 2023-09-12T14:55:09.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc6a083d640 1 -- start start 2023-09-12T14:55:09.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc6a083d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc69c0b02d0 0x7fc69c0c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:09.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc6a083d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc69c15ad60 0x7fc69c15d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:09.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc6a083d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc69c15b140 0x7fc69c157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:09.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc6a083d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c1491a0 con 0x7fc69c15ad60 2023-09-12T14:55:09.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc6a083d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c0c8670 con 0x7fc69c15b140 2023-09-12T14:55:09.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc6a083d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c15dcf0 con 0x7fc69c0b02d0 2023-09-12T14:55:09.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc699d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc69c15ad60 0x7fc69c15d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:09.405 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc699d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc69c15ad60 0x7fc69c15d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41380/0 (socket says 172.21.15.131:41380) 2023-09-12T14:55:09.405 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc699d9b640 1 -- 172.21.15.131:0/2880107174 learned_addr learned my addr 172.21.15.131:0/2880107174 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:09.405 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc69ad9d640 1 --2- 172.21.15.131:0/2880107174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc69c15b140 0x7fc69c157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:09.405 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc69a59c640 1 --2- 172.21.15.131:0/2880107174 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc69c0b02d0 0x7fc69c0c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:09.405 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc699d9b640 1 -- 172.21.15.131:0/2880107174 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc69c0b02d0 msgr2=0x7fc69c0c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:09.405 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc699d9b640 1 --2- 172.21.15.131:0/2880107174 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc69c0b02d0 0x7fc69c0c8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc699d9b640 1 -- 172.21.15.131:0/2880107174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc69c15b140 msgr2=0x7fc69c157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:09.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc699d9b640 1 --2- 172.21.15.131:0/2880107174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc69c15b140 0x7fc69c157230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc699d9b640 1 -- 172.21.15.131:0/2880107174 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc690009560 con 0x7fc69c15ad60 2023-09-12T14:55:09.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc69ad9d640 1 --2- 172.21.15.131:0/2880107174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc69c15b140 0x7fc69c157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:09.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc699d9b640 1 --2- 172.21.15.131:0/2880107174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc69c15ad60 0x7fc69c15d7b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc690000c10 tx=0x7fc6900a56f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:09.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc68affd640 1 -- 172.21.15.131:0/2880107174 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc690005230 con 0x7fc69c15ad60 2023-09-12T14:55:09.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc68affd640 1 -- 172.21.15.131:0/2880107174 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc690005480 con 0x7fc69c15ad60 2023-09-12T14:55:09.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc6a083d640 1 -- 172.21.15.131:0/2880107174 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc69c148590 con 0x7fc69c15ad60 2023-09-12T14:55:09.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc68affd640 1 -- 172.21.15.131:0/2880107174 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc690004930 con 0x7fc69c15ad60 2023-09-12T14:55:09.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.397+0000 7fc6a083d640 1 -- 172.21.15.131:0/2880107174 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc69c148b50 con 0x7fc69c15ad60 2023-09-12T14:55:09.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.401+0000 7fc6a083d640 1 -- 172.21.15.131:0/2880107174 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc69c0cac80 con 0x7fc69c15ad60 2023-09-12T14:55:09.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.401+0000 7fc68affd640 1 -- 172.21.15.131:0/2880107174 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fc6900b4070 con 0x7fc69c15ad60 2023-09-12T14:55:09.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.401+0000 7fc68affd640 1 --2- 172.21.15.131:0/2880107174 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc67c065b20 0x7fc67c067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:09.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.401+0000 7fc69a59c640 1 --2- 172.21.15.131:0/2880107174 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc67c065b20 0x7fc67c067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:09.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.401+0000 7fc68affd640 1 -- 172.21.15.131:0/2880107174 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc69011d400 con 0x7fc69c15ad60 2023-09-12T14:55:09.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.401+0000 7fc69a59c640 1 --2- 172.21.15.131:0/2880107174 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc67c065b20 0x7fc67c067fe0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc68c007fd0 tx=0x7fc68c009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:09.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.405+0000 7fc68affd640 1 -- 172.21.15.131:0/2880107174 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fc6900eacb0 con 0x7fc69c15ad60 2023-09-12T14:55:09.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.545+0000 7fc6a083d640 1 -- 172.21.15.131:0/2880107174 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc69c073580 con 0x7fc69c15ad60 2023-09-12T14:55:09.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.549+0000 7fc68affd640 1 -- 172.21.15.131:0/2880107174 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc6900a59a0 con 0x7fc69c15ad60 2023-09-12T14:55:09.550 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-09-12T14:55:09.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.549+0000 7fc6a083d640 1 -- 172.21.15.131:0/2880107174 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc67c065b20 msgr2=0x7fc67c067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:09.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.549+0000 7fc6a083d640 1 --2- 172.21.15.131:0/2880107174 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc67c065b20 0x7fc67c067fe0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc68c007fd0 tx=0x7fc68c009040 comp rx=0 tx=0).stop 2023-09-12T14:55:09.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.549+0000 7fc6a083d640 1 -- 172.21.15.131:0/2880107174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc69c15ad60 msgr2=0x7fc69c15d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:09.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.549+0000 7fc6a083d640 1 --2- 172.21.15.131:0/2880107174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc69c15ad60 0x7fc69c15d7b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc690000c10 tx=0x7fc6900a56f0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.549+0000 7fc6a083d640 1 -- 172.21.15.131:0/2880107174 shutdown_connections 2023-09-12T14:55:09.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.549+0000 7fc6a083d640 1 --2- 172.21.15.131:0/2880107174 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc67c065b20 0x7fc67c067fe0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.549+0000 7fc6a083d640 1 --2- 172.21.15.131:0/2880107174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc69c15b140 0x7fc69c157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.549+0000 7fc6a083d640 1 --2- 172.21.15.131:0/2880107174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc69c15ad60 0x7fc69c15d7b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.549+0000 7fc6a083d640 1 --2- 172.21.15.131:0/2880107174 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc69c0b02d0 0x7fc69c0c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.549+0000 7fc6a083d640 1 -- 172.21.15.131:0/2880107174 >> 172.21.15.131:0/2880107174 conn(0x7fc69c069bb0 msgr2=0x7fc69c1551c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:09.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.549+0000 7fc6a083d640 1 -- 172.21.15.131:0/2880107174 shutdown_connections 2023-09-12T14:55:09.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.549+0000 7fc6a083d640 1 -- 172.21.15.131:0/2880107174 wait complete. 2023-09-12T14:55:09.568 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T14:55:09.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2aebf59640 1 -- 172.21.15.131:0/2060194021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae406f390 msgr2=0x7f2ae406f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:09.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2aebf59640 1 --2- 172.21.15.131:0/2060194021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae406f390 0x7f2ae406f820 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2ae406eea0 tx=0x7f2adc01cd30 comp rx=0 tx=0).stop 2023-09-12T14:55:09.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2aebf59640 1 -- 172.21.15.131:0/2060194021 shutdown_connections 2023-09-12T14:55:09.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2aebf59640 1 --2- 172.21.15.131:0/2060194021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae406f390 0x7f2ae406f820 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2aebf59640 1 --2- 172.21.15.131:0/2060194021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ae4158180 0x7f2ae40cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2aebf59640 1 --2- 172.21.15.131:0/2060194021 >> v2:172.21.15.187:3300/0 conn(0x7f2ae40ab730 0x7f2ae40abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2aebf59640 1 -- 172.21.15.131:0/2060194021 >> 172.21.15.131:0/2060194021 conn(0x7f2ae4069840 msgr2=0x7f2ae40cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:09.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2aebf59640 1 -- 172.21.15.131:0/2060194021 shutdown_connections 2023-09-12T14:55:09.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2aebf59640 1 -- 172.21.15.131:0/2060194021 wait complete. 2023-09-12T14:55:09.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2aebf59640 1 Processor -- start 2023-09-12T14:55:09.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2aebf59640 1 -- start start 2023-09-12T14:55:09.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2aebf59640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ae406f390 0x7f2ae4147ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:09.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2aebf59640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ae40ab730 0x7f2ae41f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:09.696 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2aebf59640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae4158180 0x7f2ae4156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:09.696 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2aebf59640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae40b0dc0 con 0x7f2ae4158180 2023-09-12T14:55:09.696 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2aebf59640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae41481e0 con 0x7f2ae40ab730 2023-09-12T14:55:09.696 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2aebf59640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae4148410 con 0x7f2ae406f390 2023-09-12T14:55:09.696 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2aea4f6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae4158180 0x7f2ae4156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:09.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2aea4f6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae4158180 0x7f2ae4156b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41392/0 (socket says 172.21.15.131:41392) 2023-09-12T14:55:09.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2aea4f6640 1 -- 172.21.15.131:0/952433792 learned_addr learned my addr 172.21.15.131:0/952433792 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:09.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2ae9cf5640 1 --2- 172.21.15.131:0/952433792 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ae406f390 0x7f2ae4147ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:09.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2ae94f4640 1 --2- 172.21.15.131:0/952433792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ae40ab730 0x7f2ae41f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:09.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2aea4f6640 1 -- 172.21.15.131:0/952433792 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ae406f390 msgr2=0x7f2ae4147ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:09.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2aea4f6640 1 --2- 172.21.15.131:0/952433792 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ae406f390 0x7f2ae4147ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2aea4f6640 1 -- 172.21.15.131:0/952433792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ae40ab730 msgr2=0x7f2ae41f3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:09.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2aea4f6640 1 --2- 172.21.15.131:0/952433792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ae40ab730 0x7f2ae41f3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2aea4f6640 1 -- 172.21.15.131:0/952433792 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2adc0098e0 con 0x7f2ae4158180 2023-09-12T14:55:09.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.689+0000 7f2ae94f4640 1 --2- 172.21.15.131:0/952433792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ae40ab730 0x7f2ae41f3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:09.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.693+0000 7f2aea4f6640 1 --2- 172.21.15.131:0/952433792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae4158180 0x7f2ae4156b70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009bd0 tx=0x7f2adc004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:09.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.693+0000 7f2ada7fc640 1 -- 172.21.15.131:0/952433792 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc0bb070 con 0x7f2ae4158180 2023-09-12T14:55:09.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.693+0000 7f2ada7fc640 1 -- 172.21.15.131:0/952433792 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2adc003c80 con 0x7f2ae4158180 2023-09-12T14:55:09.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.693+0000 7f2aebf59640 1 -- 172.21.15.131:0/952433792 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae41486f0 con 0x7f2ae4158180 2023-09-12T14:55:09.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.693+0000 7f2ada7fc640 1 -- 172.21.15.131:0/952433792 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc0ad900 con 0x7f2ae4158180 2023-09-12T14:55:09.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.693+0000 7f2aebf59640 1 -- 172.21.15.131:0/952433792 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae4148cb0 con 0x7f2ae4158180 2023-09-12T14:55:09.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.693+0000 7f2aebf59640 1 -- 172.21.15.131:0/952433792 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ae415a8c0 con 0x7f2ae4158180 2023-09-12T14:55:09.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.693+0000 7f2ada7fc640 1 -- 172.21.15.131:0/952433792 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f2adc0b4040 con 0x7f2ae4158180 2023-09-12T14:55:09.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.693+0000 7f2ada7fc640 1 --2- 172.21.15.131:0/952433792 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2ac8065b20 0x7f2ac8067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:09.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.693+0000 7f2ada7fc640 1 -- 172.21.15.131:0/952433792 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2adc11dbc0 con 0x7f2ae4158180 2023-09-12T14:55:09.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.693+0000 7f2ae9cf5640 1 --2- 172.21.15.131:0/952433792 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2ac8065b20 0x7f2ac8067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:09.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.693+0000 7f2ae9cf5640 1 --2- 172.21.15.131:0/952433792 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2ac8065b20 0x7f2ac8067fe0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0005950 tx=0x7f2ae000a4e0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:09.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.697+0000 7f2ada7fc640 1 -- 172.21.15.131:0/952433792 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f2adc0eb5a0 con 0x7f2ae4158180 2023-09-12T14:55:09.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.837+0000 7f2aebf59640 1 -- 172.21.15.131:0/952433792 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2ae4152550 con 0x7f2ae4158180 2023-09-12T14:55:09.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.837+0000 7f2ada7fc640 1 -- 172.21.15.131:0/952433792 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2adc0b20a0 con 0x7f2ae4158180 2023-09-12T14:55:09.841 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-09-12T14:55:09.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.841+0000 7f2aebf59640 1 -- 172.21.15.131:0/952433792 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2ac8065b20 msgr2=0x7f2ac8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:09.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.841+0000 7f2aebf59640 1 --2- 172.21.15.131:0/952433792 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2ac8065b20 0x7f2ac8067fe0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0005950 tx=0x7f2ae000a4e0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.841+0000 7f2aebf59640 1 -- 172.21.15.131:0/952433792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae4158180 msgr2=0x7f2ae4156b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:09.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.841+0000 7f2aebf59640 1 --2- 172.21.15.131:0/952433792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae4158180 0x7f2ae4156b70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009bd0 tx=0x7f2adc004dc0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.841+0000 7f2aebf59640 1 -- 172.21.15.131:0/952433792 shutdown_connections 2023-09-12T14:55:09.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.841+0000 7f2aebf59640 1 --2- 172.21.15.131:0/952433792 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2ac8065b20 0x7f2ac8067fe0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.841+0000 7f2aebf59640 1 --2- 172.21.15.131:0/952433792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae4158180 0x7f2ae4156b70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.841+0000 7f2aebf59640 1 --2- 172.21.15.131:0/952433792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ae40ab730 0x7f2ae41f3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.841+0000 7f2aebf59640 1 --2- 172.21.15.131:0/952433792 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ae406f390 0x7f2ae4147ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.841+0000 7f2aebf59640 1 -- 172.21.15.131:0/952433792 >> 172.21.15.131:0/952433792 conn(0x7f2ae4069840 msgr2=0x7f2ae40acef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:09.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.841+0000 7f2aebf59640 1 -- 172.21.15.131:0/952433792 shutdown_connections 2023-09-12T14:55:09.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.841+0000 7f2aebf59640 1 -- 172.21.15.131:0/952433792 wait complete. 2023-09-12T14:55:09.859 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T14:55:09.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f7025e9b640 1 -- 172.21.15.131:0/3407019238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70200d1680 msgr2=0x7f70200b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:09.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f7025e9b640 1 --2- 172.21.15.131:0/3407019238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70200d1680 0x7f70200b0750 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7008009840 tx=0x7f700801c7d0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f7025e9b640 1 -- 172.21.15.131:0/3407019238 shutdown_connections 2023-09-12T14:55:09.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f7025e9b640 1 --2- 172.21.15.131:0/3407019238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f702015e9f0 0x7f70200b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f7025e9b640 1 --2- 172.21.15.131:0/3407019238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70200d1680 0x7f70200b0750 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f7025e9b640 1 -- 172.21.15.131:0/3407019238 >> 172.21.15.131:0/3407019238 conn(0x7f702009a200 msgr2=0x7f70200d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:09.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f7025e9b640 1 -- 172.21.15.131:0/3407019238 shutdown_connections 2023-09-12T14:55:09.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f7025e9b640 1 -- 172.21.15.131:0/3407019238 wait complete. 2023-09-12T14:55:09.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f7025e9b640 1 Processor -- start 2023-09-12T14:55:09.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f7025e9b640 1 -- start start 2023-09-12T14:55:09.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f7025e9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70200cf2d0 0x7f702014c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:09.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f7025e9b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70200d1680 0x7f702014c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:09.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f7025e9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f702015e9f0 0x7f702014ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:09.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f7025e9b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70200c6f10 con 0x7f70200cf2d0 2023-09-12T14:55:09.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f7025e9b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702014dfd0 con 0x7f702015e9f0 2023-09-12T14:55:09.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f7025e9b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f702015c4c0 con 0x7f70200d1680 2023-09-12T14:55:09.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f701f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70200cf2d0 0x7f702014c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:09.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f701f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70200cf2d0 0x7f702014c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41400/0 (socket says 172.21.15.131:41400) 2023-09-12T14:55:09.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f701f7fe640 1 -- 172.21.15.131:0/3692784449 learned_addr learned my addr 172.21.15.131:0/3692784449 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:09.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f701effd640 1 --2- 172.21.15.131:0/3692784449 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70200d1680 0x7f702014c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:09.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f701ffff640 1 --2- 172.21.15.131:0/3692784449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f702015e9f0 0x7f702014ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:09.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f701f7fe640 1 -- 172.21.15.131:0/3692784449 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70200d1680 msgr2=0x7f702014c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:09.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f701f7fe640 1 --2- 172.21.15.131:0/3692784449 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70200d1680 0x7f702014c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f701f7fe640 1 -- 172.21.15.131:0/3692784449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f702015e9f0 msgr2=0x7f702014ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:09.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f701f7fe640 1 --2- 172.21.15.131:0/3692784449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f702015e9f0 0x7f702014ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:09.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f701f7fe640 1 -- 172.21.15.131:0/3692784449 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70080094a0 con 0x7f70200cf2d0 2023-09-12T14:55:09.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f701ffff640 1 --2- 172.21.15.131:0/3692784449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f702015e9f0 0x7f702014ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:09.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f701f7fe640 1 --2- 172.21.15.131:0/3692784449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70200cf2d0 0x7f702014c160 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f701000c420 tx=0x7f701000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:09.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f7017fff640 1 -- 172.21.15.131:0/3692784449 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7010004230 con 0x7f70200cf2d0 2023-09-12T14:55:09.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f7017fff640 1 -- 172.21.15.131:0/3692784449 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f70100044a0 con 0x7f70200cf2d0 2023-09-12T14:55:09.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.969+0000 7f7017fff640 1 -- 172.21.15.131:0/3692784449 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7010005260 con 0x7f70200cf2d0 2023-09-12T14:55:09.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.973+0000 7f7025e9b640 1 -- 172.21.15.131:0/3692784449 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f702015c7a0 con 0x7f70200cf2d0 2023-09-12T14:55:09.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.973+0000 7f7025e9b640 1 -- 172.21.15.131:0/3692784449 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f702015cd60 con 0x7f70200cf2d0 2023-09-12T14:55:09.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.973+0000 7f7025e9b640 1 -- 172.21.15.131:0/3692784449 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7020073580 con 0x7f70200cf2d0 2023-09-12T14:55:09.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.973+0000 7f7017fff640 1 -- 172.21.15.131:0/3692784449 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f70100053c0 con 0x7f70200cf2d0 2023-09-12T14:55:09.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.973+0000 7f7017fff640 1 --2- 172.21.15.131:0/3692784449 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7000065b20 0x7f7000067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:09.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.973+0000 7f7017fff640 1 -- 172.21.15.131:0/3692784449 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70100830f0 con 0x7f70200cf2d0 2023-09-12T14:55:09.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.973+0000 7f701effd640 1 --2- 172.21.15.131:0/3692784449 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7000065b20 0x7f7000067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:09.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.973+0000 7f701effd640 1 --2- 172.21.15.131:0/3692784449 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7000065b20 0x7f7000067fe0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f70080af000 tx=0x7f700801b010 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:09.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:09.977+0000 7f7017fff640 1 -- 172.21.15.131:0/3692784449 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f7010050b20 con 0x7f70200cf2d0 2023-09-12T14:55:10.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.149+0000 7f7025e9b640 1 -- 172.21.15.131:0/3692784449 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70200afee0 con 0x7f70200cf2d0 2023-09-12T14:55:10.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.149+0000 7f7017fff640 1 -- 172.21.15.131:0/3692784449 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f7010020020 con 0x7f70200cf2d0 2023-09-12T14:55:10.152 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:10.152 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4344,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/491327908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":491327908},{"type":"v1","addr":"172.21.15.131:6837","nonce":491327908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3470212105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3470212105},{"type":"v1","addr":"172.21.15.187:6833","nonce":3470212105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/91478708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":91478708},{"type":"v1","addr":"172.21.15.131:6836","nonce":91478708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-12T14:54:58.030444+0000","modified":"2023-09-12T14:55:00.236338+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4396":{"gid":4396,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/290727181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":290727181},{"type":"v1","addr":"172.21.15.187:6835","nonce":290727181}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-12T14:55:10.153 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 6 2023-09-12T14:55:10.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.153+0000 7f7025e9b640 1 -- 172.21.15.131:0/3692784449 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7000065b20 msgr2=0x7f7000067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:10.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.153+0000 7f7025e9b640 1 --2- 172.21.15.131:0/3692784449 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7000065b20 0x7f7000067fe0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f70080af000 tx=0x7f700801b010 comp rx=0 tx=0).stop 2023-09-12T14:55:10.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.153+0000 7f7025e9b640 1 -- 172.21.15.131:0/3692784449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70200cf2d0 msgr2=0x7f702014c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:10.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.153+0000 7f7025e9b640 1 --2- 172.21.15.131:0/3692784449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70200cf2d0 0x7f702014c160 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f701000c420 tx=0x7f701000c8e0 comp rx=0 tx=0).stop 2023-09-12T14:55:10.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.153+0000 7f7025e9b640 1 -- 172.21.15.131:0/3692784449 shutdown_connections 2023-09-12T14:55:10.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.153+0000 7f7025e9b640 1 --2- 172.21.15.131:0/3692784449 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7000065b20 0x7f7000067fe0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:10.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.153+0000 7f7025e9b640 1 --2- 172.21.15.131:0/3692784449 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70200d1680 0x7f702014c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:10.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.153+0000 7f7025e9b640 1 --2- 172.21.15.131:0/3692784449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f702015e9f0 0x7f702014ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:10.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.153+0000 7f7025e9b640 1 --2- 172.21.15.131:0/3692784449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70200cf2d0 0x7f702014c160 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:10.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.153+0000 7f7025e9b640 1 -- 172.21.15.131:0/3692784449 >> 172.21.15.131:0/3692784449 conn(0x7f702009a200 msgr2=0x7f702006b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:10.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.153+0000 7f7025e9b640 1 -- 172.21.15.131:0/3692784449 shutdown_connections 2023-09-12T14:55:10.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.153+0000 7f7025e9b640 1 -- 172.21.15.131:0/3692784449 wait complete. 2023-09-12T14:55:10.169 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:55:10.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc05eade640 1 -- 172.21.15.131:0/1163357629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc05806db90 msgr2=0x7fc058070a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:10.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc05eade640 1 --2- 172.21.15.131:0/1163357629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc05806db90 0x7fc058070a80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc0580713f0 tx=0x7fc04c01c7d0 comp rx=0 tx=0).stop 2023-09-12T14:55:10.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc05eade640 1 -- 172.21.15.131:0/1163357629 shutdown_connections 2023-09-12T14:55:10.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc05eade640 1 --2- 172.21.15.131:0/1163357629 >> v2:172.21.15.187:3300/0 conn(0x7fc0580afa40 0x7fc0580afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:10.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc05eade640 1 --2- 172.21.15.131:0/1163357629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc05806db90 0x7fc058070a80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:10.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc05eade640 1 -- 172.21.15.131:0/1163357629 >> 172.21.15.131:0/1163357629 conn(0x7fc0580cdd80 msgr2=0x7fc0580ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:10.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc05eade640 1 -- 172.21.15.131:0/1163357629 shutdown_connections 2023-09-12T14:55:10.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc05eade640 1 -- 172.21.15.131:0/1163357629 wait complete. 2023-09-12T14:55:10.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc05eade640 1 Processor -- start 2023-09-12T14:55:10.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc05eade640 1 -- start start 2023-09-12T14:55:10.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc05eade640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc05806db90 0x7fc058149850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:10.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc05eade640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0580afa40 0x7fc05814d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:10.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc05eade640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc058157bc0 0x7fc058151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:10.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc05eade640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05814dd20 con 0x7fc0580afa40 2023-09-12T14:55:10.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc05eade640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05814c810 con 0x7fc058157bc0 2023-09-12T14:55:10.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc05eade640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc05814bf20 con 0x7fc05806db90 2023-09-12T14:55:10.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc057fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0580afa40 0x7fc05814d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:10.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc057fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0580afa40 0x7fc05814d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41422/0 (socket says 172.21.15.131:41422) 2023-09-12T14:55:10.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc057fff640 1 -- 172.21.15.131:0/1764756901 learned_addr learned my addr 172.21.15.131:0/1764756901 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:10.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc05d07b640 1 --2- 172.21.15.131:0/1764756901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc058157bc0 0x7fc058151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:10.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc057fff640 1 -- 172.21.15.131:0/1764756901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc05806db90 msgr2=0x7fc058149850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:10.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc057fff640 1 --2- 172.21.15.131:0/1764756901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc05806db90 0x7fc058149850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:10.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc057fff640 1 -- 172.21.15.131:0/1764756901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc058157bc0 msgr2=0x7fc058151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:10.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc057fff640 1 --2- 172.21.15.131:0/1764756901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc058157bc0 0x7fc058151ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:10.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc057fff640 1 -- 172.21.15.131:0/1764756901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04c009560 con 0x7fc0580afa40 2023-09-12T14:55:10.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc057fff640 1 --2- 172.21.15.131:0/1764756901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0580afa40 0x7fc05814d260 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc04c000c10 tx=0x7fc04c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc0557fa640 1 -- 172.21.15.131:0/1764756901 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04c0044f0 con 0x7fc0580afa40 2023-09-12T14:55:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc0557fa640 1 -- 172.21.15.131:0/1764756901 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc04c005160 con 0x7fc0580afa40 2023-09-12T14:55:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc05eade640 1 -- 172.21.15.131:0/1764756901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc05814c1c0 con 0x7fc0580afa40 2023-09-12T14:55:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc0557fa640 1 -- 172.21.15.131:0/1764756901 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04c005510 con 0x7fc0580afa40 2023-09-12T14:55:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc05eade640 1 -- 172.21.15.131:0/1764756901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc05814a310 con 0x7fc0580afa40 2023-09-12T14:55:10.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.285+0000 7fc05eade640 1 -- 172.21.15.131:0/1764756901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc058073580 con 0x7fc0580afa40 2023-09-12T14:55:10.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.289+0000 7fc0557fa640 1 -- 172.21.15.131:0/1764756901 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fc04c0b4070 con 0x7fc0580afa40 2023-09-12T14:55:10.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.289+0000 7fc0557fa640 1 --2- 172.21.15.131:0/1764756901 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc044065b50 0x7fc044068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:10.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.289+0000 7fc0557fa640 1 -- 172.21.15.131:0/1764756901 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc04c11d2a0 con 0x7fc0580afa40 2023-09-12T14:55:10.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.289+0000 7fc05c87a640 1 --2- 172.21.15.131:0/1764756901 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc044065b50 0x7fc044068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:10.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.289+0000 7fc05c87a640 1 --2- 172.21.15.131:0/1764756901 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc044065b50 0x7fc044068010 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc048009670 tx=0x7fc048006bd0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:10.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.293+0000 7fc0557fa640 1 -- 172.21.15.131:0/1764756901 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fc04c0eac80 con 0x7fc0580afa40 2023-09-12T14:55:10.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.437+0000 7fc05eade640 1 -- 172.21.15.131:0/1764756901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc058070fc0 con 0x7fc0580afa40 2023-09-12T14:55:10.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.437+0000 7fc0557fa640 1 -- 172.21.15.131:0/1764756901 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fc058070fc0 con 0x7fc0580afa40 2023-09-12T14:55:10.440 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:10.441 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:54:59.694778+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:54:55.689983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T14:54:57.029447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:55:10.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.441+0000 7fc05eade640 1 -- 172.21.15.131:0/1764756901 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc044065b50 msgr2=0x7fc044068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:10.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.441+0000 7fc05eade640 1 --2- 172.21.15.131:0/1764756901 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc044065b50 0x7fc044068010 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc048009670 tx=0x7fc048006bd0 comp rx=0 tx=0).stop 2023-09-12T14:55:10.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.441+0000 7fc05eade640 1 -- 172.21.15.131:0/1764756901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0580afa40 msgr2=0x7fc05814d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:10.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.441+0000 7fc05eade640 1 --2- 172.21.15.131:0/1764756901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0580afa40 0x7fc05814d260 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc04c000c10 tx=0x7fc04c004040 comp rx=0 tx=0).stop 2023-09-12T14:55:10.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.441+0000 7fc05eade640 1 -- 172.21.15.131:0/1764756901 shutdown_connections 2023-09-12T14:55:10.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.441+0000 7fc05eade640 1 --2- 172.21.15.131:0/1764756901 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc044065b50 0x7fc044068010 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:10.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.441+0000 7fc05eade640 1 --2- 172.21.15.131:0/1764756901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc058157bc0 0x7fc058151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:10.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.441+0000 7fc05eade640 1 --2- 172.21.15.131:0/1764756901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0580afa40 0x7fc05814d260 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:10.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.441+0000 7fc05eade640 1 --2- 172.21.15.131:0/1764756901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc05806db90 0x7fc058149850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:10.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.441+0000 7fc05eade640 1 -- 172.21.15.131:0/1764756901 >> 172.21.15.131:0/1764756901 conn(0x7fc0580cdd80 msgr2=0x7fc058154a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:10.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.441+0000 7fc05eade640 1 -- 172.21.15.131:0/1764756901 shutdown_connections 2023-09-12T14:55:10.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.441+0000 7fc05eade640 1 -- 172.21.15.131:0/1764756901 wait complete. 2023-09-12T14:55:10.458 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T14:55:10.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.577+0000 7fb2dd60e640 1 -- 172.21.15.131:0/1047408118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d80c98d0 msgr2=0x7fb2d80af9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:10.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.577+0000 7fb2dd60e640 1 --2- 172.21.15.131:0/1047408118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d80c98d0 0x7fb2d80af9b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009c80 tx=0x7fb2c801cd10 comp rx=0 tx=0).stop 2023-09-12T14:55:10.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.577+0000 7fb2dd60e640 1 -- 172.21.15.131:0/1047408118 shutdown_connections 2023-09-12T14:55:10.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.577+0000 7fb2dd60e640 1 --2- 172.21.15.131:0/1047408118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d80c98d0 0x7fb2d80af9b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:10.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.577+0000 7fb2dd60e640 1 --2- 172.21.15.131:0/1047408118 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2d80b2600 0x7fb2d80c9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:10.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.577+0000 7fb2dd60e640 1 -- 172.21.15.131:0/1047408118 >> 172.21.15.131:0/1047408118 conn(0x7fb2d8069840 msgr2=0x7fb2d80cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:10.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.577+0000 7fb2dd60e640 1 -- 172.21.15.131:0/1047408118 shutdown_connections 2023-09-12T14:55:10.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.577+0000 7fb2dd60e640 1 -- 172.21.15.131:0/1047408118 wait complete. 2023-09-12T14:55:10.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.581+0000 7fb2dd60e640 1 Processor -- start 2023-09-12T14:55:10.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.581+0000 7fb2dd60e640 1 -- start start 2023-09-12T14:55:10.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.581+0000 7fb2dd60e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d80b2600 0x7fb2d81549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:10.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.581+0000 7fb2dd60e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2d80c98d0 0x7fb2d8150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:10.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.581+0000 7fb2dd60e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2d81574d0 0x7fb2d8143500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:10.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.581+0000 7fb2dd60e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d8144920 con 0x7fb2d80b2600 2023-09-12T14:55:10.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.581+0000 7fb2dd60e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d8155090 con 0x7fb2d80c98d0 2023-09-12T14:55:10.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.581+0000 7fb2dd60e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d8155370 con 0x7fb2d81574d0 2023-09-12T14:55:10.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.581+0000 7fb2d6ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d80b2600 0x7fb2d81549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:10.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.581+0000 7fb2d6ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d80b2600 0x7fb2d81549b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41442/0 (socket says 172.21.15.131:41442) 2023-09-12T14:55:10.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.581+0000 7fb2d6ffd640 1 -- 172.21.15.131:0/3587662384 learned_addr learned my addr 172.21.15.131:0/3587662384 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:10.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.581+0000 7fb2d6ffd640 1 -- 172.21.15.131:0/3587662384 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2d81574d0 msgr2=0x7fb2d8143500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:10.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.581+0000 7fb2d6ffd640 1 --2- 172.21.15.131:0/3587662384 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2d81574d0 0x7fb2d8143500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:10.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.581+0000 7fb2d6ffd640 1 -- 172.21.15.131:0/3587662384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2d80c98d0 msgr2=0x7fb2d8150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:10.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.581+0000 7fb2d6ffd640 1 --2- 172.21.15.131:0/3587662384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2d80c98d0 0x7fb2d8150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:10.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.581+0000 7fb2d6ffd640 1 -- 172.21.15.131:0/3587662384 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c80098e0 con 0x7fb2d80b2600 2023-09-12T14:55:10.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.581+0000 7fb2d6ffd640 1 --2- 172.21.15.131:0/3587662384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d80b2600 0x7fb2d81549b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000b410 tx=0x7fb2d000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:10.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.581+0000 7fb2cf7fe640 1 -- 172.21.15.131:0/3587662384 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0004020 con 0x7fb2d80b2600 2023-09-12T14:55:10.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.581+0000 7fb2cf7fe640 1 -- 172.21.15.131:0/3587662384 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb2d0004270 con 0x7fb2d80b2600 2023-09-12T14:55:10.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.581+0000 7fb2dd60e640 1 -- 172.21.15.131:0/3587662384 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d8155650 con 0x7fb2d80b2600 2023-09-12T14:55:10.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.581+0000 7fb2cf7fe640 1 -- 172.21.15.131:0/3587662384 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0011890 con 0x7fb2d80b2600 2023-09-12T14:55:10.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.581+0000 7fb2dd60e640 1 -- 172.21.15.131:0/3587662384 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d8150570 con 0x7fb2d80b2600 2023-09-12T14:55:10.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.581+0000 7fb2dd60e640 1 -- 172.21.15.131:0/3587662384 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d8155890 con 0x7fb2d80b2600 2023-09-12T14:55:10.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.581+0000 7fb2cf7fe640 1 -- 172.21.15.131:0/3587662384 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fb2d0003600 con 0x7fb2d80b2600 2023-09-12T14:55:10.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.581+0000 7fb2cf7fe640 1 --2- 172.21.15.131:0/3587662384 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb2b8065b20 0x7fb2b8067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:10.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.581+0000 7fb2d67fc640 1 --2- 172.21.15.131:0/3587662384 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb2b8065b20 0x7fb2b8067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:10.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.581+0000 7fb2cf7fe640 1 -- 172.21.15.131:0/3587662384 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2d0083390 con 0x7fb2d80b2600 2023-09-12T14:55:10.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.585+0000 7fb2d67fc640 1 --2- 172.21.15.131:0/3587662384 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb2b8065b20 0x7fb2b8067fe0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8155e30 tx=0x7fb2c0005c00 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:10.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.585+0000 7fb2cf7fe640 1 -- 172.21.15.131:0/3587662384 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fb2d0050c50 con 0x7fb2d80b2600 2023-09-12T14:55:10.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.761+0000 7fb2dd60e640 1 -- 172.21.15.131:0/3587662384 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2d8073580 con 0x7fb2d80b2600 2023-09-12T14:55:10.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.761+0000 7fb2cf7fe640 1 -- 172.21.15.131:0/3587662384 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4907 (secure 0 0 0) 0x7fb2d0054270 con 0x7fb2d80b2600 2023-09-12T14:55:10.764 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:10.764 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4344,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/491327908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":491327908},{"type":"v1","addr":"172.21.15.131:6837","nonce":491327908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3470212105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3470212105},{"type":"v1","addr":"172.21.15.187:6833","nonce":3470212105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/91478708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":91478708},{"type":"v1","addr":"172.21.15.131:6836","nonce":91478708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-12T14:54:58.030444+0000","modified":"2023-09-12T14:55:00.236338+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4396":{"gid":4396,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/290727181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":290727181},{"type":"v1","addr":"172.21.15.187:6835","nonce":290727181}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-12T14:55:10.765 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 6 2023-09-12T14:55:10.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.765+0000 7fb2dd60e640 1 -- 172.21.15.131:0/3587662384 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb2b8065b20 msgr2=0x7fb2b8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:10.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.765+0000 7fb2dd60e640 1 --2- 172.21.15.131:0/3587662384 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb2b8065b20 0x7fb2b8067fe0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8155e30 tx=0x7fb2c0005c00 comp rx=0 tx=0).stop 2023-09-12T14:55:10.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.765+0000 7fb2dd60e640 1 -- 172.21.15.131:0/3587662384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d80b2600 msgr2=0x7fb2d81549b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:10.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.765+0000 7fb2dd60e640 1 --2- 172.21.15.131:0/3587662384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d80b2600 0x7fb2d81549b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000b410 tx=0x7fb2d000b8d0 comp rx=0 tx=0).stop 2023-09-12T14:55:10.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.765+0000 7fb2dd60e640 1 -- 172.21.15.131:0/3587662384 shutdown_connections 2023-09-12T14:55:10.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.765+0000 7fb2dd60e640 1 --2- 172.21.15.131:0/3587662384 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb2b8065b20 0x7fb2b8067fe0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:10.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.765+0000 7fb2dd60e640 1 --2- 172.21.15.131:0/3587662384 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2d81574d0 0x7fb2d8143500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:10.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.765+0000 7fb2dd60e640 1 --2- 172.21.15.131:0/3587662384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2d80c98d0 0x7fb2d8150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:10.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.765+0000 7fb2dd60e640 1 --2- 172.21.15.131:0/3587662384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d80b2600 0x7fb2d81549b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:10.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.765+0000 7fb2dd60e640 1 -- 172.21.15.131:0/3587662384 >> 172.21.15.131:0/3587662384 conn(0x7fb2d8069840 msgr2=0x7fb2d806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:10.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.765+0000 7fb2dd60e640 1 -- 172.21.15.131:0/3587662384 shutdown_connections 2023-09-12T14:55:10.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:10.765+0000 7fb2dd60e640 1 -- 172.21.15.131:0/3587662384 wait complete. 2023-09-12T14:55:11.185 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-12T14:54:58.030444+0000', 'modified': '2023-09-12T14:55:00.236338+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4396}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4396': {'gid': 4396, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.187:6835/290727181', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6834', 'nonce': 290727181}, {'type': 'v1', 'addr': '172.21.15.187:6835', 'nonce': 290727181}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-12T14:55:11.185 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-12T14:55:11.186 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-09-12T14:55:11.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.317+0000 7fe0c61c1640 1 -- 172.21.15.131:0/1377548042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c006f390 msgr2=0x7fe0c006f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:11.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.317+0000 7fe0c61c1640 1 --2- 172.21.15.131:0/1377548042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c006f390 0x7fe0c006f820 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe0c006eea0 tx=0x7fe0b801c870 comp rx=0 tx=0).stop 2023-09-12T14:55:11.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.317+0000 7fe0c61c1640 1 -- 172.21.15.131:0/1377548042 shutdown_connections 2023-09-12T14:55:11.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.317+0000 7fe0c61c1640 1 --2- 172.21.15.131:0/1377548042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c006f390 0x7fe0c006f820 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.317+0000 7fe0c61c1640 1 --2- 172.21.15.131:0/1377548042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0c0158180 0x7fe0c00cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.317+0000 7fe0c61c1640 1 --2- 172.21.15.131:0/1377548042 >> v2:172.21.15.187:3300/0 conn(0x7fe0c00ab730 0x7fe0c00abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.317+0000 7fe0c61c1640 1 -- 172.21.15.131:0/1377548042 >> 172.21.15.131:0/1377548042 conn(0x7fe0c0069840 msgr2=0x7fe0c00cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:11.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.317+0000 7fe0c61c1640 1 -- 172.21.15.131:0/1377548042 shutdown_connections 2023-09-12T14:55:11.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.317+0000 7fe0c61c1640 1 -- 172.21.15.131:0/1377548042 wait complete. 2023-09-12T14:55:11.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.317+0000 7fe0c61c1640 1 Processor -- start 2023-09-12T14:55:11.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.317+0000 7fe0c61c1640 1 -- start start 2023-09-12T14:55:11.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.321+0000 7fe0c61c1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0c006f390 0x7fe0c00aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:11.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.321+0000 7fe0c61c1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0c00ab730 0x7fe0c00b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:11.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.321+0000 7fe0c61c1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c0158180 0x7fe0c014e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:11.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.321+0000 7fe0c61c1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c00ab4c0 con 0x7fe0c0158180 2023-09-12T14:55:11.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.321+0000 7fe0c61c1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c00ab1e0 con 0x7fe0c006f390 2023-09-12T14:55:11.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.321+0000 7fe0c61c1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c00b01a0 con 0x7fe0c00ab730 2023-09-12T14:55:11.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.321+0000 7fe0bffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c0158180 0x7fe0c014e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:11.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.321+0000 7fe0bffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c0158180 0x7fe0c014e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41470/0 (socket says 172.21.15.131:41470) 2023-09-12T14:55:11.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.321+0000 7fe0bffff640 1 -- 172.21.15.131:0/674523036 learned_addr learned my addr 172.21.15.131:0/674523036 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:11.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.321+0000 7fe0bf7fe640 1 --2- 172.21.15.131:0/674523036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0c006f390 0x7fe0c00aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:11.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.321+0000 7fe0bffff640 1 -- 172.21.15.131:0/674523036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0c00ab730 msgr2=0x7fe0c00b0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:11.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.321+0000 7fe0bffff640 1 --2- 172.21.15.131:0/674523036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0c00ab730 0x7fe0c00b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.321+0000 7fe0bffff640 1 -- 172.21.15.131:0/674523036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0c006f390 msgr2=0x7fe0c00aa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:11.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.321+0000 7fe0bffff640 1 --2- 172.21.15.131:0/674523036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0c006f390 0x7fe0c00aa8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.321+0000 7fe0bffff640 1 -- 172.21.15.131:0/674523036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b8009560 con 0x7fe0c0158180 2023-09-12T14:55:11.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.321+0000 7fe0bffff640 1 --2- 172.21.15.131:0/674523036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c0158180 0x7fe0c014e0d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8009850 tx=0x7fe0b80a5770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:11.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.321+0000 7fe0a7fff640 1 -- 172.21.15.131:0/674523036 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8003800 con 0x7fe0c0158180 2023-09-12T14:55:11.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.321+0000 7fe0a7fff640 1 -- 172.21.15.131:0/674523036 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe0b8003960 con 0x7fe0c0158180 2023-09-12T14:55:11.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.321+0000 7fe0c61c1640 1 -- 172.21.15.131:0/674523036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c00b0480 con 0x7fe0c0158180 2023-09-12T14:55:11.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.321+0000 7fe0a7fff640 1 -- 172.21.15.131:0/674523036 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8003d30 con 0x7fe0c0158180 2023-09-12T14:55:11.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.321+0000 7fe0c61c1640 1 -- 172.21.15.131:0/674523036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c00aae20 con 0x7fe0c0158180 2023-09-12T14:55:11.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.321+0000 7fe0a7fff640 1 -- 172.21.15.131:0/674523036 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fe0b80b4040 con 0x7fe0c0158180 2023-09-12T14:55:11.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.321+0000 7fe0a7fff640 1 --2- 172.21.15.131:0/674523036 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe0ac065b20 0x7fe0ac067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:11.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.321+0000 7fe0a7fff640 1 -- 172.21.15.131:0/674523036 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0b811e100 con 0x7fe0c0158180 2023-09-12T14:55:11.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.321+0000 7fe0bf7fe640 1 --2- 172.21.15.131:0/674523036 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe0ac065b20 0x7fe0ac067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:11.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.321+0000 7fe0bf7fe640 1 --2- 172.21.15.131:0/674523036 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe0ac065b20 0x7fe0ac067fe0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe0c01f7610 tx=0x7fe0b0008230 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:11.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0c92ee000 1 Processor -- start 2023-09-12T14:55:11.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0c92ee000 1 -- start start 2023-09-12T14:55:11.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0c92ee000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ce861ed0 0x55d1ce8622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:11.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0c92ee000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 0x55d1ce817df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:11.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0c92ee000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ce818330 0x55d1ce818710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:11.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0c92ee000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1ce66f510 con 0x55d1ce8627f0 2023-09-12T14:55:11.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0c92ee000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1ce66ca40 con 0x55d1ce861ed0 2023-09-12T14:55:11.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0c92ee000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d1ce668330 con 0x55d1ce818330 2023-09-12T14:55:11.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0beffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 0x55d1ce817df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:11.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0beffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 0x55d1ce817df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41480/0 (socket says 172.21.15.131:41480) 2023-09-12T14:55:11.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0beffd640 1 -- 172.21.15.131:0/394157779 learned_addr learned my addr 172.21.15.131:0/394157779 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:11.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0bf7fe640 1 --2- 172.21.15.131:0/394157779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ce861ed0 0x55d1ce8622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:11.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0bffff640 1 --2- 172.21.15.131:0/394157779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ce818330 0x55d1ce818710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:11.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0beffd640 1 -- 172.21.15.131:0/394157779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ce818330 msgr2=0x55d1ce818710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T14:55:11.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0beffd640 1 --2- 172.21.15.131:0/394157779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ce818330 0x55d1ce818710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0beffd640 1 -- 172.21.15.131:0/394157779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ce861ed0 msgr2=0x55d1ce8622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T14:55:11.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0beffd640 1 --2- 172.21.15.131:0/394157779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ce861ed0 0x55d1ce8622b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0beffd640 1 -- 172.21.15.131:0/394157779 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1ce362890 con 0x55d1ce8627f0 2023-09-12T14:55:11.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0bffff640 1 --2- 172.21.15.131:0/394157779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ce818330 0x55d1ce818710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:11.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0beffd640 1 --2- 172.21.15.131:0/394157779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 0x55d1ce817df0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe0b400a8d0 tx=0x7fe0b400ad90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77dd33ccd79bdfb server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:11.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0a57fa640 1 -- 172.21.15.131:0/394157779 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b400ccc0 con 0x55d1ce8627f0 2023-09-12T14:55:11.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0a57fa640 1 -- 172.21.15.131:0/394157779 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe0b400f450 con 0x55d1ce8627f0 2023-09-12T14:55:11.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0a57fa640 1 -- 172.21.15.131:0/394157779 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b400f860 con 0x55d1ce8627f0 2023-09-12T14:55:11.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0c92ee000 1 -- 172.21.15.131:0/394157779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 msgr2=0x55d1ce817df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:11.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0c92ee000 1 --2- 172.21.15.131:0/394157779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 0x55d1ce817df0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe0b400a8d0 tx=0x7fe0b400ad90 comp rx=0 tx=0).stop 2023-09-12T14:55:11.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0c92ee000 1 -- 172.21.15.131:0/394157779 shutdown_connections 2023-09-12T14:55:11.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0c92ee000 1 --2- 172.21.15.131:0/394157779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ce818330 0x55d1ce818710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0c92ee000 1 --2- 172.21.15.131:0/394157779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 0x55d1ce817df0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0c92ee000 1 --2- 172.21.15.131:0/394157779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ce861ed0 0x55d1ce8622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0c92ee000 1 -- 172.21.15.131:0/394157779 >> 172.21.15.131:0/394157779 conn(0x55d1ce81e490 msgr2=0x55d1ce8166d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:11.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0c92ee000 1 -- 172.21.15.131:0/394157779 shutdown_connections 2023-09-12T14:55:11.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0c92ee000 1 -- 172.21.15.131:0/394157779 wait complete. 2023-09-12T14:55:11.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0c92ee000 1 -- start start 2023-09-12T14:55:11.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0c92ee000 1 Processor -- start 2023-09-12T14:55:11.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0c92ee000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ce818330 0x55d1ce822e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:11.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0c92ee000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ce861ed0 0x55d1ce8233d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:11.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0c92ee000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 0x55d1ce823910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:11.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0c92ee000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1ce622ed0 con 0x55d1ce8627f0 2023-09-12T14:55:11.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0c92ee000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1ce622b70 con 0x55d1ce818330 2023-09-12T14:55:11.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0c92ee000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d1ce323940 con 0x55d1ce861ed0 2023-09-12T14:55:11.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.325+0000 7fe0c92ee000 10 client.0 _ll_register_callbacks cb 0x55d1ce7364a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T14:55:11.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe0bffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 0x55d1ce823910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:11.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe0bffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 0x55d1ce823910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41482/0 (socket says 172.21.15.131:41482) 2023-09-12T14:55:11.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe0bffff640 1 -- 172.21.15.131:0/2331658413 learned_addr learned my addr 172.21.15.131:0/2331658413 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:11.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe0beffd640 1 --2- 172.21.15.131:0/2331658413 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ce861ed0 0x55d1ce8233d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:11.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe0bf7fe640 1 --2- 172.21.15.131:0/2331658413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ce818330 0x55d1ce822e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:11.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe0bffff640 1 -- 172.21.15.131:0/2331658413 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ce861ed0 msgr2=0x55d1ce8233d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T14:55:11.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe0bffff640 1 --2- 172.21.15.131:0/2331658413 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ce861ed0 0x55d1ce8233d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe0bffff640 1 -- 172.21.15.131:0/2331658413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ce818330 msgr2=0x55d1ce822e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T14:55:11.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe0bffff640 1 --2- 172.21.15.131:0/2331658413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ce818330 0x55d1ce822e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe0bffff640 1 -- 172.21.15.131:0/2331658413 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b400a500 con 0x55d1ce8627f0 2023-09-12T14:55:11.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe0bf7fe640 1 --2- 172.21.15.131:0/2331658413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ce818330 0x55d1ce822e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:11.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe0beffd640 1 --2- 172.21.15.131:0/2331658413 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ce861ed0 0x55d1ce8233d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:11.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe0bffff640 1 --2- 172.21.15.131:0/2331658413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 0x55d1ce823910 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe0b80ebce0 tx=0x7fe0b80ef6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143daf22ea73df88 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:11.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe09e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-12T14:55:11.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe09e7fc640 1 -- 172.21.15.131:0/2331658413 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b80f3610 con 0x55d1ce8627f0 2023-09-12T14:55:11.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe09e7fc640 1 -- 172.21.15.131:0/2331658413 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0b80f1ea0 con 0x55d1ce8627f0 2023-09-12T14:55:11.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe0c92ee000 1 -- 172.21.15.131:0/2331658413 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1ce61f880 con 0x55d1ce8627f0 2023-09-12T14:55:11.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe09e7fc640 1 -- 172.21.15.131:0/2331658413 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0b80f77c0 con 0x55d1ce8627f0 2023-09-12T14:55:11.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe0c92ee000 10 client.4701 fetch_fsmap learned FSMap version 6 2023-09-12T14:55:11.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe0c92ee000 1 -- 172.21.15.131:0/2331658413 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55d1ce62b290 con 0x55d1ce8627f0 2023-09-12T14:55:11.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe09e7fc640 1 -- 172.21.15.131:0/2331658413 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x55d1ce61f880 con 0x55d1ce8627f0 2023-09-12T14:55:11.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe09e7fc640 1 -- 172.21.15.131:0/2331658413 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 6) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe0b80c6020 con 0x55d1ce8627f0 2023-09-12T14:55:11.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe0c92ee000 10 client.4701 fetch_fsmap finished waiting for FSMap version 6 2023-09-12T14:55:11.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe0c92ee000 10 client.4701 resolve_mds: resolved name 'd' to daemon mds.d 2023-09-12T14:55:11.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe0c92ee000 20 client.4701 populate_metadata read hostname 'smithi131' 2023-09-12T14:55:11.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe0c92ee000 1 --2- 172.21.15.131:0/2331658413 >> [v2:172.21.15.187:6834/290727181,v1:172.21.15.187:6835/290727181] conn(0x55d1ce871170 0x55d1ce873560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:11.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe0c92ee000 4 client.4701 mds_command: new command op to 4396 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T14:55:11.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe0c92ee000 1 -- 172.21.15.131:0/2331658413 --> [v2:172.21.15.187:6834/290727181,v1:172.21.15.187:6835/290727181] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d1ce628700 con 0x55d1ce871170 2023-09-12T14:55:11.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe0bf7fe640 1 --2- 172.21.15.131:0/2331658413 >> [v2:172.21.15.187:6834/290727181,v1:172.21.15.187:6835/290727181] conn(0x55d1ce871170 0x55d1ce873560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:11.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe09e7fc640 10 client.4701 ms_handle_connect on v2:172.21.15.187:6834/290727181 2023-09-12T14:55:11.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe0bf7fe640 1 --2- 172.21.15.131:0/2331658413 >> [v2:172.21.15.187:6834/290727181,v1:172.21.15.187:6835/290727181] conn(0x55d1ce871170 0x55d1ce873560 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dabc8ebbb7aa15f server_cookie=1aa3b6a5f9ca1af6 in_seq=0 out_seq=0 2023-09-12T14:55:11.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe09e7fc640 1 -- 172.21.15.131:0/2331658413 <== mds.0 v2:172.21.15.187:6834/290727181 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d1ce628700 con 0x55d1ce871170 2023-09-12T14:55:11.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe09e7fc640 10 client.4701 handle_command_reply: tid=0 2023-09-12T14:55:11.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.329+0000 7fe0c92ee000 1 client.4701 shutdown 2023-09-12T14:55:11.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.333+0000 7fe0c92ee000 1 -- 172.21.15.131:0/2331658413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 msgr2=0x55d1ce823910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:11.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.333+0000 7fe0c92ee000 1 --2- 172.21.15.131:0/2331658413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 0x55d1ce823910 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe0b80ebce0 tx=0x7fe0b80ef6b0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.333+0000 7fe0c92ee000 1 -- 172.21.15.131:0/2331658413 shutdown_connections 2023-09-12T14:55:11.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.333+0000 7fe0c92ee000 1 --2- 172.21.15.131:0/2331658413 >> [v2:172.21.15.187:6834/290727181,v1:172.21.15.187:6835/290727181] conn(0x55d1ce871170 0x55d1ce873560 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.333+0000 7fe0c92ee000 1 --2- 172.21.15.131:0/2331658413 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ce861ed0 0x55d1ce8233d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.333+0000 7fe0c92ee000 1 --2- 172.21.15.131:0/2331658413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 0x55d1ce823910 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.333+0000 7fe09e7fc640 0 client.4701 ms_handle_reset on v2:172.21.15.187:6834/290727181 2023-09-12T14:55:11.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.333+0000 7fe0c92ee000 1 --2- 172.21.15.131:0/2331658413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ce818330 0x55d1ce822e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.333+0000 7fe0c92ee000 1 -- 172.21.15.131:0/2331658413 >> 172.21.15.131:0/2331658413 conn(0x55d1ce81e490 msgr2=0x55d1ce816dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:11.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.333+0000 7fe0c92ee000 1 -- 172.21.15.131:0/2331658413 shutdown_connections 2023-09-12T14:55:11.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.333+0000 7fe0c92ee000 1 -- 172.21.15.131:0/2331658413 wait complete. 2023-09-12T14:55:11.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.333+0000 7fe0c92ee000 20 client.4701 trim_cache size 0 max 16384 2023-09-12T14:55:11.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.345+0000 7fe0c92ee000 1 Processor -- start 2023-09-12T14:55:11.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.345+0000 7fe0c92ee000 1 -- start start 2023-09-12T14:55:11.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ce818330 0x55d1ce8bc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:11.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ce861ed0 0x55d1ce8bc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:11.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 0x55d1ce8bcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:11.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1ce603db0 con 0x55d1ce8627f0 2023-09-12T14:55:11.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1ce66ae90 con 0x55d1ce818330 2023-09-12T14:55:11.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d1ce820100 con 0x55d1ce861ed0 2023-09-12T14:55:11.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0bffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 0x55d1ce8bcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:11.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0bffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 0x55d1ce8bcdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41496/0 (socket says 172.21.15.131:41496) 2023-09-12T14:55:11.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0bffff640 1 -- 172.21.15.131:0/293794855 learned_addr learned my addr 172.21.15.131:0/293794855 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:11.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0bffff640 1 -- 172.21.15.131:0/293794855 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ce861ed0 msgr2=0x55d1ce8bc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T14:55:11.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0bffff640 1 --2- 172.21.15.131:0/293794855 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ce861ed0 0x55d1ce8bc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0bffff640 1 -- 172.21.15.131:0/293794855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ce818330 msgr2=0x55d1ce8bc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T14:55:11.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0bffff640 1 --2- 172.21.15.131:0/293794855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ce818330 0x55d1ce8bc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0bffff640 1 -- 172.21.15.131:0/293794855 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1ce611500 con 0x55d1ce8627f0 2023-09-12T14:55:11.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0bffff640 1 --2- 172.21.15.131:0/293794855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 0x55d1ce8bcdc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe0b801d110 tx=0x7fe0b811c590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9898a283b74aee3a server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:11.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0a5ffb640 1 -- 172.21.15.131:0/293794855 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b80f3610 con 0x55d1ce8627f0 2023-09-12T14:55:11.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0a5ffb640 1 -- 172.21.15.131:0/293794855 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe0b80f7330 con 0x55d1ce8627f0 2023-09-12T14:55:11.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0a5ffb640 1 -- 172.21.15.131:0/293794855 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b80eef20 con 0x55d1ce8627f0 2023-09-12T14:55:11.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 1 -- 172.21.15.131:0/293794855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 msgr2=0x55d1ce8bcdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:11.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 1 --2- 172.21.15.131:0/293794855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 0x55d1ce8bcdc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe0b801d110 tx=0x7fe0b811c590 comp rx=0 tx=0).stop 2023-09-12T14:55:11.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 1 -- 172.21.15.131:0/293794855 shutdown_connections 2023-09-12T14:55:11.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 1 --2- 172.21.15.131:0/293794855 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ce861ed0 0x55d1ce8bc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 1 --2- 172.21.15.131:0/293794855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 0x55d1ce8bcdc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 1 --2- 172.21.15.131:0/293794855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ce818330 0x55d1ce8bc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 1 -- 172.21.15.131:0/293794855 >> 172.21.15.131:0/293794855 conn(0x55d1ce81e490 msgr2=0x55d1ce873df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:11.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 1 -- 172.21.15.131:0/293794855 shutdown_connections 2023-09-12T14:55:11.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 1 -- 172.21.15.131:0/293794855 wait complete. 2023-09-12T14:55:11.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 1 -- start start 2023-09-12T14:55:11.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 1 Processor -- start 2023-09-12T14:55:11.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ce818330 0x55d1ce8603b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:11.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ce861ed0 0x55d1ce8bc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:11.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 0x55d1ce8bca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:11.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1ce622ed0 con 0x55d1ce8627f0 2023-09-12T14:55:11.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1ce811150 con 0x55d1ce861ed0 2023-09-12T14:55:11.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d1ce81b110 con 0x55d1ce818330 2023-09-12T14:55:11.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 10 client.0 _ll_register_callbacks cb 0x55d1ce607e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T14:55:11.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0bffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 0x55d1ce8bca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:11.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0bffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 0x55d1ce8bca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41504/0 (socket says 172.21.15.131:41504) 2023-09-12T14:55:11.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0bffff640 1 -- 172.21.15.131:0/2681453057 learned_addr learned my addr 172.21.15.131:0/2681453057 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:11.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0beffd640 1 --2- 172.21.15.131:0/2681453057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ce861ed0 0x55d1ce8bc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:11.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0bffff640 1 -- 172.21.15.131:0/2681453057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ce818330 msgr2=0x55d1ce8603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T14:55:11.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0bffff640 1 --2- 172.21.15.131:0/2681453057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ce818330 0x55d1ce8603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0bffff640 1 -- 172.21.15.131:0/2681453057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ce861ed0 msgr2=0x55d1ce8bc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T14:55:11.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0bffff640 1 --2- 172.21.15.131:0/2681453057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ce861ed0 0x55d1ce8bc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0bffff640 1 -- 172.21.15.131:0/2681453057 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1ce62b290 con 0x55d1ce8627f0 2023-09-12T14:55:11.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0bffff640 1 --2- 172.21.15.131:0/2681453057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 0x55d1ce8bca90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe0b80a9040 tx=0x7fe0b811c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343cec62469374db server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:11.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe09e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-12T14:55:11.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe09e7fc640 1 -- 172.21.15.131:0/2681453057 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b80eef20 con 0x55d1ce8627f0 2023-09-12T14:55:11.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe09e7fc640 1 -- 172.21.15.131:0/2681453057 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0b80f77c0 con 0x55d1ce8627f0 2023-09-12T14:55:11.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 1 -- 172.21.15.131:0/2681453057 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1ce66e310 con 0x55d1ce8627f0 2023-09-12T14:55:11.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe09e7fc640 1 -- 172.21.15.131:0/2681453057 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0b80f3610 con 0x55d1ce8627f0 2023-09-12T14:55:11.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 10 client.4707 fetch_fsmap learned FSMap version 6 2023-09-12T14:55:11.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 1 -- 172.21.15.131:0/2681453057 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55d1ce362cc0 con 0x55d1ce8627f0 2023-09-12T14:55:11.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe09e7fc640 1 -- 172.21.15.131:0/2681453057 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x55d1ce66e310 con 0x55d1ce8627f0 2023-09-12T14:55:11.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe09e7fc640 1 -- 172.21.15.131:0/2681453057 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 6) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe0b80c6070 con 0x55d1ce8627f0 2023-09-12T14:55:11.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 10 client.4707 fetch_fsmap finished waiting for FSMap version 6 2023-09-12T14:55:11.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 10 client.4707 resolve_mds: resolved name 'd' to daemon mds.d 2023-09-12T14:55:11.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 20 client.4707 populate_metadata read hostname 'smithi131' 2023-09-12T14:55:11.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 1 --2- 172.21.15.131:0/2681453057 >> [v2:172.21.15.187:6834/290727181,v1:172.21.15.187:6835/290727181] conn(0x55d1ce8c0880 0x55d1ce8c0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:11.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 4 client.4707 mds_command: new command op to 4396 tid=0[{"prefix": "status"}] 2023-09-12T14:55:11.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.349+0000 7fe0c92ee000 1 -- 172.21.15.131:0/2681453057 --> [v2:172.21.15.187:6834/290727181,v1:172.21.15.187:6835/290727181] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d1ce661310 con 0x55d1ce8c0880 2023-09-12T14:55:11.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe0bf7fe640 1 --2- 172.21.15.131:0/2681453057 >> [v2:172.21.15.187:6834/290727181,v1:172.21.15.187:6835/290727181] conn(0x55d1ce8c0880 0x55d1ce8c0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:11.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe09e7fc640 10 client.4707 ms_handle_connect on v2:172.21.15.187:6834/290727181 2023-09-12T14:55:11.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe0bf7fe640 1 --2- 172.21.15.131:0/2681453057 >> [v2:172.21.15.187:6834/290727181,v1:172.21.15.187:6835/290727181] conn(0x55d1ce8c0880 0x55d1ce8c0c60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3103c24717b8973 server_cookie=28a43426c627739d in_seq=0 out_seq=0 2023-09-12T14:55:11.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe09e7fc640 1 -- 172.21.15.131:0/2681453057 <== mds.0 v2:172.21.15.187:6834/290727181 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x55d1ce661310 con 0x55d1ce8c0880 2023-09-12T14:55:11.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe09e7fc640 10 client.4707 handle_command_reply: tid=0 2023-09-12T14:55:11.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe0c92ee000 1 client.4707 shutdown 2023-09-12T14:55:11.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe0c92ee000 1 -- 172.21.15.131:0/2681453057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 msgr2=0x55d1ce8bca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:11.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe0c92ee000 1 --2- 172.21.15.131:0/2681453057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 0x55d1ce8bca90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe0b80a9040 tx=0x7fe0b811c6d0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe0c92ee000 1 -- 172.21.15.131:0/2681453057 shutdown_connections 2023-09-12T14:55:11.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe0c92ee000 1 --2- 172.21.15.131:0/2681453057 >> [v2:172.21.15.187:6834/290727181,v1:172.21.15.187:6835/290727181] conn(0x55d1ce8c0880 0x55d1ce8c0c60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe0c92ee000 1 --2- 172.21.15.131:0/2681453057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1ce8627f0 0x55d1ce8bca90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe0c92ee000 1 --2- 172.21.15.131:0/2681453057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1ce861ed0 0x55d1ce8bc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe0c92ee000 1 --2- 172.21.15.131:0/2681453057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d1ce818330 0x55d1ce8603b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe0c92ee000 1 -- 172.21.15.131:0/2681453057 >> 172.21.15.131:0/2681453057 conn(0x55d1ce81e490 msgr2=0x55d1ce8744f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:11.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe09e7fc640 0 client.4707 ms_handle_reset on v2:172.21.15.187:6834/290727181 2023-09-12T14:55:11.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe0c92ee000 1 -- 172.21.15.131:0/2681453057 shutdown_connections 2023-09-12T14:55:11.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe0c92ee000 1 -- 172.21.15.131:0/2681453057 wait complete. 2023-09-12T14:55:11.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe0c92ee000 20 client.4707 trim_cache size 0 max 16384 2023-09-12T14:55:11.373 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-09-12T14:55:11.373 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "6efb91e0-47b4-40bb-ae12-7c7ea0e893df", 2023-09-12T14:55:11.374 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2023-09-12T14:55:11.374 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 4396, 2023-09-12T14:55:11.374 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2023-09-12T14:55:11.374 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2023-09-12T14:55:11.374 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2023-09-12T14:55:11.375 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 12.472493511, 2023-09-12T14:55:11.375 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 6, 2023-09-12T14:55:11.375 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 20, 2023-09-12T14:55:11.375 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 20, 2023-09-12T14:55:11.375 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 20.145433982 2023-09-12T14:55:11.376 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-09-12T14:55:11.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe0c61c1640 1 -- 172.21.15.131:0/674523036 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe0ac065b20 msgr2=0x7fe0ac067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:11.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe0c61c1640 1 --2- 172.21.15.131:0/674523036 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe0ac065b20 0x7fe0ac067fe0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe0c01f7610 tx=0x7fe0b0008230 comp rx=0 tx=0).stop 2023-09-12T14:55:11.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe0c61c1640 1 -- 172.21.15.131:0/674523036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c0158180 msgr2=0x7fe0c014e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:11.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe0c61c1640 1 --2- 172.21.15.131:0/674523036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c0158180 0x7fe0c014e0d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8009850 tx=0x7fe0b80a5770 comp rx=0 tx=0).stop 2023-09-12T14:55:11.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe0c61c1640 1 -- 172.21.15.131:0/674523036 shutdown_connections 2023-09-12T14:55:11.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe0c61c1640 1 --2- 172.21.15.131:0/674523036 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe0ac065b20 0x7fe0ac067fe0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe0c61c1640 1 --2- 172.21.15.131:0/674523036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0c00ab730 0x7fe0c00b0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe0c61c1640 1 --2- 172.21.15.131:0/674523036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c0158180 0x7fe0c014e0d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe0c61c1640 1 --2- 172.21.15.131:0/674523036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0c006f390 0x7fe0c00aa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.379 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe0c61c1640 1 -- 172.21.15.131:0/674523036 >> 172.21.15.131:0/674523036 conn(0x7fe0c0069840 msgr2=0x7fe0c006bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:11.379 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe0c61c1640 1 -- 172.21.15.131:0/674523036 shutdown_connections 2023-09-12T14:55:11.379 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.353+0000 7fe0c61c1640 1 -- 172.21.15.131:0/674523036 wait complete. 2023-09-12T14:55:11.379 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-09-12T14:55:11.401 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-09-12T14:55:11.402 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-09-12T14:55:11.402 INFO:tasks.ceph_fuse:client.0 config is {} 2023-09-12T14:55:11.402 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-09-12T14:55:11.403 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-12T14:55:11.403 INFO:tasks.ceph_fuse:client.2 config is {} 2023-09-12T14:55:11.403 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-09-12T14:55:11.404 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-09-12T14:55:11.404 INFO:tasks.ceph_fuse:client.1 config is {} 2023-09-12T14:55:11.404 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-09-12T14:55:11.404 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-09-12T14:55:11.405 INFO:tasks.ceph_fuse:client.3 config is {} 2023-09-12T14:55:11.405 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-09-12T14:55:11.405 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-09-12T14:55:11.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:11.406 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2023-09-12T14:55:11.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:11.411 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link delete ceph-brx 2023-09-12T14:55:11.466 INFO:teuthology.orchestra.run.smithi131.stderr:Cannot find device "ceph-brx" 2023-09-12T14:55:11.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T14:55:11.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:11.468 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2023-09-12T14:55:11.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:11.475 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link delete ceph-brx 2023-09-12T14:55:11.535 INFO:teuthology.orchestra.run.smithi187.stderr:Cannot find device "ceph-brx" 2023-09-12T14:55:11.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T14:55:11.537 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-09-12T14:55:11.538 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-12T14:55:11.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.661+0000 7f03d8f10640 1 -- 172.21.15.131:0/125996122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03bc0058e0 msgr2=0x7f03bc005d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:11.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.661+0000 7f03d8f10640 1 --2- 172.21.15.131:0/125996122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03bc0058e0 0x7f03bc005d60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f03c800c260 tx=0x7f03c801f740 comp rx=0 tx=0).stop 2023-09-12T14:55:11.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.661+0000 7f03d8f10640 1 -- 172.21.15.131:0/125996122 shutdown_connections 2023-09-12T14:55:11.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.661+0000 7f03d8f10640 1 --2- 172.21.15.131:0/125996122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03bc0058e0 0x7f03bc005d60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.661+0000 7f03d8f10640 1 -- 172.21.15.131:0/125996122 >> 172.21.15.131:0/125996122 conn(0x7f03d4069840 msgr2=0x7f03d40cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:11.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.661+0000 7f03d8f10640 1 -- 172.21.15.131:0/125996122 shutdown_connections 2023-09-12T14:55:11.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.661+0000 7f03d8f10640 1 -- 172.21.15.131:0/125996122 wait complete. 2023-09-12T14:55:11.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.661+0000 7f03d8f10640 1 Processor -- start 2023-09-12T14:55:11.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.661+0000 7f03d8f10640 1 -- start start 2023-09-12T14:55:11.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.661+0000 7f03d8f10640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03bc0058e0 0x7f03d41ecef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:11.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.661+0000 7f03d8f10640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03d414dd20 0x7f03d4152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:11.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.661+0000 7f03d8f10640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03d414c5f0 0x7f03d414e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:11.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.661+0000 7f03d8f10640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d41ed7b0 con 0x7f03d414c5f0 2023-09-12T14:55:11.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.661+0000 7f03d8f10640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d41eda90 con 0x7f03d414dd20 2023-09-12T14:55:11.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.661+0000 7f03d8f10640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d41edd70 con 0x7f03bc0058e0 2023-09-12T14:55:11.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.661+0000 7f03d2d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03d414c5f0 0x7f03d414e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:11.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.661+0000 7f03d2d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03d414c5f0 0x7f03d414e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41522/0 (socket says 172.21.15.131:41522) 2023-09-12T14:55:11.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.661+0000 7f03d2d9d640 1 -- 172.21.15.131:0/1199770812 learned_addr learned my addr 172.21.15.131:0/1199770812 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:11.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.665+0000 7f03d1d9b640 1 --2- 172.21.15.131:0/1199770812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03d414dd20 0x7f03d4152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:11.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.665+0000 7f03d259c640 1 --2- 172.21.15.131:0/1199770812 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03bc0058e0 0x7f03d41ecef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:11.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.665+0000 7f03d2d9d640 1 -- 172.21.15.131:0/1199770812 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03bc0058e0 msgr2=0x7f03d41ecef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:11.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.665+0000 7f03d2d9d640 1 --2- 172.21.15.131:0/1199770812 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03bc0058e0 0x7f03d41ecef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.665+0000 7f03d2d9d640 1 -- 172.21.15.131:0/1199770812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03d414dd20 msgr2=0x7f03d4152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:11.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.665+0000 7f03d2d9d640 1 --2- 172.21.15.131:0/1199770812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03d414dd20 0x7f03d4152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.665+0000 7f03d2d9d640 1 -- 172.21.15.131:0/1199770812 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d40cbe00 con 0x7f03d414c5f0 2023-09-12T14:55:11.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.665+0000 7f03d259c640 1 --2- 172.21.15.131:0/1199770812 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03bc0058e0 0x7f03d41ecef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:55:11.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.665+0000 7f03d1d9b640 1 --2- 172.21.15.131:0/1199770812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03d414dd20 0x7f03d4152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:55:11.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.665+0000 7f03d2d9d640 1 --2- 172.21.15.131:0/1199770812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03d414c5f0 0x7f03d414e280 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f03d4070940 tx=0x7f03cc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:11.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.665+0000 7f03c2ffd640 1 -- 172.21.15.131:0/1199770812 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc003b80 con 0x7f03d414c5f0 2023-09-12T14:55:11.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.665+0000 7f03c2ffd640 1 -- 172.21.15.131:0/1199770812 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f03cc003df0 con 0x7f03d414c5f0 2023-09-12T14:55:11.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.665+0000 7f03d8f10640 1 -- 172.21.15.131:0/1199770812 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d414bdf0 con 0x7f03d414c5f0 2023-09-12T14:55:11.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.665+0000 7f03c2ffd640 1 -- 172.21.15.131:0/1199770812 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc005330 con 0x7f03d414c5f0 2023-09-12T14:55:11.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.665+0000 7f03d8f10640 1 -- 172.21.15.131:0/1199770812 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d41ed430 con 0x7f03d414c5f0 2023-09-12T14:55:11.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.665+0000 7f03d8f10640 1 -- 172.21.15.131:0/1199770812 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d4073580 con 0x7f03d414c5f0 2023-09-12T14:55:11.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.665+0000 7f03c2ffd640 1 -- 172.21.15.131:0/1199770812 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f03cc005490 con 0x7f03d414c5f0 2023-09-12T14:55:11.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.665+0000 7f03c2ffd640 1 --2- 172.21.15.131:0/1199770812 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f03b4065b50 0x7f03b4068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:11.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.665+0000 7f03d259c640 1 --2- 172.21.15.131:0/1199770812 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f03b4065b50 0x7f03b4068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:11.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.665+0000 7f03c2ffd640 1 -- 172.21.15.131:0/1199770812 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03cc082a40 con 0x7f03d414c5f0 2023-09-12T14:55:11.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.665+0000 7f03d259c640 1 --2- 172.21.15.131:0/1199770812 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f03b4065b50 0x7f03b4068010 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f03c40058d0 tx=0x7f03c4005860 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:11.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.669+0000 7f03c2ffd640 1 -- 172.21.15.131:0/1199770812 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f03cc084e60 con 0x7f03d414c5f0 2023-09-12T14:55:11.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.853+0000 7f03d8f10640 1 -- 172.21.15.131:0/1199770812 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f03d414c2a0 con 0x7f03d414c5f0 2023-09-12T14:55:11.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.853+0000 7f03c2ffd640 1 -- 172.21.15.131:0/1199770812 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f03cc0503a0 con 0x7f03d414c5f0 2023-09-12T14:55:11.856 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-12T14:55:11.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.857+0000 7f03d8f10640 1 -- 172.21.15.131:0/1199770812 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f03b4065b50 msgr2=0x7f03b4068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:11.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.857+0000 7f03d8f10640 1 --2- 172.21.15.131:0/1199770812 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f03b4065b50 0x7f03b4068010 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f03c40058d0 tx=0x7f03c4005860 comp rx=0 tx=0).stop 2023-09-12T14:55:11.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.857+0000 7f03d8f10640 1 -- 172.21.15.131:0/1199770812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03d414c5f0 msgr2=0x7f03d414e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:11.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.857+0000 7f03d8f10640 1 --2- 172.21.15.131:0/1199770812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03d414c5f0 0x7f03d414e280 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f03d4070940 tx=0x7f03cc00c8e0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.857+0000 7f03d8f10640 1 -- 172.21.15.131:0/1199770812 shutdown_connections 2023-09-12T14:55:11.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.857+0000 7f03d8f10640 1 --2- 172.21.15.131:0/1199770812 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f03b4065b50 0x7f03b4068010 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.857+0000 7f03d8f10640 1 --2- 172.21.15.131:0/1199770812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03d414c5f0 0x7f03d414e280 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.857+0000 7f03d8f10640 1 --2- 172.21.15.131:0/1199770812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03d414dd20 0x7f03d4152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.857+0000 7f03d8f10640 1 --2- 172.21.15.131:0/1199770812 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03bc0058e0 0x7f03d41ecef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:11.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.857+0000 7f03d8f10640 1 -- 172.21.15.131:0/1199770812 >> 172.21.15.131:0/1199770812 conn(0x7f03d4069840 msgr2=0x7f03d406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:11.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.857+0000 7f03d8f10640 1 -- 172.21.15.131:0/1199770812 shutdown_connections 2023-09-12T14:55:11.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:11.857+0000 7f03d8f10640 1 -- 172.21.15.131:0/1199770812 wait complete. 2023-09-12T14:55:11.873 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-12T14:55:11.874 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-12T14:55:11.874 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2023-09-12T14:55:11.874 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-12T14:55:11.874 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-12T14:55:11.875 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-12T14:55:11.875 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-12T14:55:11.875 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-12T14:55:11.875 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-12T14:55:11.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:11.876 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2023-09-12T14:55:11.881 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-12T14:55:11.882 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-12T14:55:11.882 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2023-09-12T14:55:11.882 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:11.882 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2023-09-12T14:55:11.882 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:11.883 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-12T14:55:11.883 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:55:11.883 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2023-09-12T14:55:11.883 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-12T14:55:11.884 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:55:11.884 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2023-09-12T14:55:11.884 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-12T14:55:11.884 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:55:11.884 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-12T14:55:11.885 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:55:11.885 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-09-12T14:55:11.885 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42484sec preferred_lft 42484sec 2023-09-12T14:55:11.885 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2023-09-12T14:55:11.885 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:11.885 INFO:teuthology.orchestra.run.smithi131.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-09-12T14:55:11.886 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 02:42:21:fb:7e:42 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:55:11.886 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-09-12T14:55:11.886 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:11.887 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-09-12T14:55:11.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:11.887 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-09-12T14:55:11.888 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-09-12T14:55:11.888 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add name ceph-brx type bridge 2023-09-12T14:55:11.888 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip addr flush dev ceph-brx 2023-09-12T14:55:11.888 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set ceph-brx up 2023-09-12T14:55:11.889 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-09-12T14:55:11.889 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-09-12T14:55:11.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:11.959 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-12T14:55:12.009 INFO:teuthology.orchestra.run.smithi131.stdout:1 2023-09-12T14:55:12.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:12.011 DEBUG:teuthology.orchestra.run.smithi131:> route 2023-09-12T14:55:12.236 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2023-09-12T14:55:12.237 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-12T14:55:12.237 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-09-12T14:55:12.237 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-12T14:55:12.237 INFO:teuthology.orchestra.run.smithi131.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-09-12T14:55:12.238 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-12T14:55:12.238 INFO:teuthology.orchestra.run.smithi131.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-12T14:55:12.238 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-12T14:55:12.238 INFO:teuthology.orchestra.run.smithi131.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-12T14:55:12.239 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-12T14:55:12.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:12.239 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-09-12T14:55:12.240 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-09-12T14:55:12.240 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-12T14:55:12.240 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-12T14:55:12.241 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-12T14:55:12.241 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-09-12T14:55:12.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:12.276 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2023-09-12T14:55:12.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:12.324 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list-id 2023-09-12T14:55:12.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:12.371 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-09-12T14:55:12.371 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-09-12T14:55:12.372 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-12T14:55:12.372 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-09-12T14:55:12.372 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-09-12T14:55:12.441 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-09-12T14:55:12.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:12.441 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-09-12T14:55:12.442 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-09-12T14:55:12.442 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-09-12T14:55:12.442 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-09-12T14:55:12.443 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-09-12T14:55:12.443 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-09-12T14:55:12.443 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-09-12T14:55:12.444 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-09-12T14:55:12.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:12.551 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-09-12T14:55:12.551 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-09-12T14:55:12.552 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2023-09-12T14:55:12.552 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set dev brx.0 master ceph-brx 2023-09-12T14:55:12.552 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-09-12T14:55:12.571 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-12T14:55:12.572 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-12T14:55:12.572 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-12T14:55:12.618 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-12T14:55:12.618 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-12T14:55:12.618 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-12T14:55:12.670 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2023-09-12T14:55:12.727 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-12T14:55:12.728 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-12T14:55:12.784 INFO:teuthology.orchestra.run.smithi131.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-12T14:55:12.785 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-12T14:55:12.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:12.786 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2023-09-12T14:55:12.833 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-12T14:55:12.834 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-12T14:55:12.879 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2023-09-12T14:55:12.892 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-12T14:55:12.893 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-12T14:55:12.943 INFO:teuthology.orchestra.run.smithi131.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-12T14:55:12.944 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-12T14:55:12.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:12.945 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2023-09-12T14:55:12.978 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:2023-09-12T14:55:12.977+0000 7f1833fec580 -1 init, newargv = 0x5617455c48d0 newargc=15 2023-09-12T14:55:12.978 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[46027]: starting ceph client 2023-09-12T14:55:12.985 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[46027]: starting fuse 2023-09-12T14:55:14.083 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2023-09-12T14:55:14.100 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-12T14:55:14.100 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-12T14:55:14.153 INFO:teuthology.orchestra.run.smithi131.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-12T14:55:14.154 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-12T14:55:14.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:14.155 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2023-09-12T14:55:14.207 INFO:teuthology.orchestra.run.smithi131.stdout:45 2023-09-12T14:55:14.207 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-09-12T14:55:14.207 DEBUG:teuthology.orchestra.run.smithi131:> sudo stdin-killer -- python3 -c ' 2023-09-12T14:55:14.208 DEBUG:teuthology.orchestra.run.smithi131:> import glob 2023-09-12T14:55:14.208 DEBUG:teuthology.orchestra.run.smithi131:> import re 2023-09-12T14:55:14.208 DEBUG:teuthology.orchestra.run.smithi131:> import os 2023-09-12T14:55:14.209 DEBUG:teuthology.orchestra.run.smithi131:> import subprocess 2023-09-12T14:55:14.209 DEBUG:teuthology.orchestra.run.smithi131:> 2023-09-12T14:55:14.209 DEBUG:teuthology.orchestra.run.smithi131:> def _find_admin_socket(client_name): 2023-09-12T14:55:14.210 DEBUG:teuthology.orchestra.run.smithi131:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-12T14:55:14.210 DEBUG:teuthology.orchestra.run.smithi131:> files = glob.glob(asok_path) 2023-09-12T14:55:14.210 DEBUG:teuthology.orchestra.run.smithi131:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-12T14:55:14.211 DEBUG:teuthology.orchestra.run.smithi131:> 2023-09-12T14:55:14.211 DEBUG:teuthology.orchestra.run.smithi131:> # Given a non-glob path, it better be there 2023-09-12T14:55:14.211 DEBUG:teuthology.orchestra.run.smithi131:> if "*" not in asok_path: 2023-09-12T14:55:14.211 DEBUG:teuthology.orchestra.run.smithi131:> assert(len(files) == 1) 2023-09-12T14:55:14.212 DEBUG:teuthology.orchestra.run.smithi131:> return files[0] 2023-09-12T14:55:14.212 DEBUG:teuthology.orchestra.run.smithi131:> 2023-09-12T14:55:14.213 DEBUG:teuthology.orchestra.run.smithi131:> for f in files: 2023-09-12T14:55:14.213 DEBUG:teuthology.orchestra.run.smithi131:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-12T14:55:14.213 DEBUG:teuthology.orchestra.run.smithi131:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-12T14:55:14.213 DEBUG:teuthology.orchestra.run.smithi131:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-12T14:55:14.214 DEBUG:teuthology.orchestra.run.smithi131:> contents = proc_f.read() 2023-09-12T14:55:14.214 DEBUG:teuthology.orchestra.run.smithi131:> if mountpoint in contents: 2023-09-12T14:55:14.214 DEBUG:teuthology.orchestra.run.smithi131:> return f 2023-09-12T14:55:14.215 DEBUG:teuthology.orchestra.run.smithi131:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-12T14:55:14.215 DEBUG:teuthology.orchestra.run.smithi131:> 2023-09-12T14:55:14.215 DEBUG:teuthology.orchestra.run.smithi131:> print(_find_admin_socket("client.0")) 2023-09-12T14:55:14.216 DEBUG:teuthology.orchestra.run.smithi131:> ' 2023-09-12T14:55:14.328 INFO:teuthology.orchestra.run.smithi131.stdout:/var/run/ceph/ceph-client.0.46027.asok 2023-09-12T14:55:14.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-12T14:55:14.340 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46027.asok 2023-09-12T14:55:14.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:14.341 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46027.asok status 2023-09-12T14:55:14.494 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-09-12T14:55:14.494 INFO:teuthology.orchestra.run.smithi131.stdout: "metadata": { 2023-09-12T14:55:14.494 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_sha1": "2110e00747b31a0d73768c9e5229da18b26b8aa0", 2023-09-12T14:55:14.494 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 18.0.0-6088-g2110e007 (2110e00747b31a0d73768c9e5229da18b26b8aa0) reef (dev)", 2023-09-12T14:55:14.495 INFO:teuthology.orchestra.run.smithi131.stdout: "entity_id": "0", 2023-09-12T14:55:14.495 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": "smithi131", 2023-09-12T14:55:14.495 INFO:teuthology.orchestra.run.smithi131.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-12T14:55:14.495 INFO:teuthology.orchestra.run.smithi131.stdout: "pid": "46027", 2023-09-12T14:55:14.495 INFO:teuthology.orchestra.run.smithi131.stdout: "root": "/" 2023-09-12T14:55:14.496 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:14.496 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_count": 0, 2023-09-12T14:55:14.496 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_pinned_count": 0, 2023-09-12T14:55:14.496 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 4716, 2023-09-12T14:55:14.496 INFO:teuthology.orchestra.run.smithi131.stdout: "inst": { 2023-09-12T14:55:14.497 INFO:teuthology.orchestra.run.smithi131.stdout: "name": { 2023-09-12T14:55:14.497 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "client", 2023-09-12T14:55:14.497 INFO:teuthology.orchestra.run.smithi131.stdout: "num": 4716 2023-09-12T14:55:14.497 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:14.497 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2023-09-12T14:55:14.497 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2023-09-12T14:55:14.498 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2023-09-12T14:55:14.498 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 2974821816 2023-09-12T14:55:14.498 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:14.498 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:14.498 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2023-09-12T14:55:14.499 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2023-09-12T14:55:14.499 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2023-09-12T14:55:14.499 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 2974821816 2023-09-12T14:55:14.499 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:14.499 INFO:teuthology.orchestra.run.smithi131.stdout: "inst_str": "client.4716 192.168.0.1:0/2974821816", 2023-09-12T14:55:14.500 INFO:teuthology.orchestra.run.smithi131.stdout: "addr_str": "192.168.0.1:0/2974821816", 2023-09-12T14:55:14.500 INFO:teuthology.orchestra.run.smithi131.stdout: "inode_count": 1, 2023-09-12T14:55:14.500 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_epoch": 6, 2023-09-12T14:55:14.500 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch": 20, 2023-09-12T14:55:14.500 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch_barrier": 0, 2023-09-12T14:55:14.500 INFO:teuthology.orchestra.run.smithi131.stdout: "blocklisted": false, 2023-09-12T14:55:14.501 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs" 2023-09-12T14:55:14.501 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-09-12T14:55:14.508 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-12T14:55:14.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f836c6b1640 1 -- 172.21.15.131:0/924580713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83500039b0 msgr2=0x7f8350003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:14.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f836c6b1640 1 --2- 172.21.15.131:0/924580713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83500039b0 0x7f8350003dd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f836000bb10 tx=0x7f836001cd50 comp rx=0 tx=0).stop 2023-09-12T14:55:14.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f836ac4e640 1 -- 172.21.15.131:0/924580713 reap_dead start 2023-09-12T14:55:14.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f836c6b1640 1 -- 172.21.15.131:0/924580713 shutdown_connections 2023-09-12T14:55:14.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f836c6b1640 1 -- 172.21.15.131:0/924580713 >> 172.21.15.131:0/924580713 conn(0x7f836409a200 msgr2=0x7f83640d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:14.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f836c6b1640 1 -- 172.21.15.131:0/924580713 shutdown_connections 2023-09-12T14:55:14.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f836c6b1640 1 -- 172.21.15.131:0/924580713 wait complete. 2023-09-12T14:55:14.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f836c6b1640 1 Processor -- start 2023-09-12T14:55:14.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f836c6b1640 1 -- start start 2023-09-12T14:55:14.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f836c6b1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83500039b0 0x7f836409e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:14.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f836c6b1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f836406fed0 0x7f836415a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:14.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f836c6b1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83640cc5f0 0x7f83640cca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:14.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f836c6b1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836415b6f0 con 0x7f836406fed0 2023-09-12T14:55:14.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f836c6b1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83640ccfb0 con 0x7f83640cc5f0 2023-09-12T14:55:14.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f836c6b1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f836415ade0 con 0x7f83500039b0 2023-09-12T14:55:14.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f8369c4c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f836406fed0 0x7f836415a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:14.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f8369c4c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f836406fed0 0x7f836415a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41534/0 (socket says 172.21.15.131:41534) 2023-09-12T14:55:14.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f8369c4c640 1 -- 172.21.15.131:0/4264097925 learned_addr learned my addr 172.21.15.131:0/4264097925 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:14.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f836a44d640 1 --2- 172.21.15.131:0/4264097925 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83500039b0 0x7f836409e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:14.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f836ac4e640 1 --2- 172.21.15.131:0/4264097925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83640cc5f0 0x7f83640cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:14.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f8369c4c640 1 -- 172.21.15.131:0/4264097925 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83500039b0 msgr2=0x7f836409e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:14.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f8369c4c640 1 --2- 172.21.15.131:0/4264097925 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83500039b0 0x7f836409e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:14.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f8369c4c640 1 -- 172.21.15.131:0/4264097925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83640cc5f0 msgr2=0x7f83640cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:14.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f8369c4c640 1 --2- 172.21.15.131:0/4264097925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83640cc5f0 0x7f83640cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:14.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f8369c4c640 1 -- 172.21.15.131:0/4264097925 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836409ebf0 con 0x7f836406fed0 2023-09-12T14:55:14.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f836ac4e640 1 --2- 172.21.15.131:0/4264097925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83640cc5f0 0x7f83640cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:14.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f8369c4c640 1 --2- 172.21.15.131:0/4264097925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f836406fed0 0x7f836415a620 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8354006960 tx=0x7f8354001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:14.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f835affd640 1 -- 172.21.15.131:0/4264097925 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8354011870 con 0x7f836406fed0 2023-09-12T14:55:14.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f835affd640 1 -- 172.21.15.131:0/4264097925 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8354011ac0 con 0x7f836406fed0 2023-09-12T14:55:14.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f836c6b1640 1 -- 172.21.15.131:0/4264097925 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f836415b030 con 0x7f836406fed0 2023-09-12T14:55:14.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f835affd640 1 -- 172.21.15.131:0/4264097925 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8354001d50 con 0x7f836406fed0 2023-09-12T14:55:14.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.633+0000 7f836c6b1640 1 -- 172.21.15.131:0/4264097925 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f836415ab60 con 0x7f836406fed0 2023-09-12T14:55:14.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.637+0000 7f836c6b1640 1 -- 172.21.15.131:0/4264097925 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f836415b830 con 0x7f836406fed0 2023-09-12T14:55:14.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.637+0000 7f835affd640 1 -- 172.21.15.131:0/4264097925 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f8354024440 con 0x7f836406fed0 2023-09-12T14:55:14.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.637+0000 7f835affd640 1 --2- 172.21.15.131:0/4264097925 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8348065b20 0x7f8348067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:14.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.637+0000 7f835affd640 1 -- 172.21.15.131:0/4264097925 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8354084390 con 0x7f836406fed0 2023-09-12T14:55:14.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.637+0000 7f836a44d640 1 --2- 172.21.15.131:0/4264097925 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8348065b20 0x7f8348067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:14.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.637+0000 7f836a44d640 1 --2- 172.21.15.131:0/4264097925 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8348065b20 0x7f8348067fe0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f836000bb10 tx=0x7f83600a7040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:14.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.641+0000 7f835affd640 1 -- 172.21.15.131:0/4264097925 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f8354051e50 con 0x7f836406fed0 2023-09-12T14:55:14.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.833+0000 7f836c6b1640 1 -- 172.21.15.131:0/4264097925 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f836415b830 con 0x7f836406fed0 2023-09-12T14:55:14.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.833+0000 7f835affd640 1 -- 172.21.15.131:0/4264097925 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f836415b830 con 0x7f836406fed0 2023-09-12T14:55:14.835 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-12T14:55:14.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.833+0000 7f836c6b1640 1 -- 172.21.15.131:0/4264097925 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8348065b20 msgr2=0x7f8348067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:14.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.833+0000 7f836c6b1640 1 --2- 172.21.15.131:0/4264097925 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8348065b20 0x7f8348067fe0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f836000bb10 tx=0x7f83600a7040 comp rx=0 tx=0).stop 2023-09-12T14:55:14.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.833+0000 7f836c6b1640 1 -- 172.21.15.131:0/4264097925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f836406fed0 msgr2=0x7f836415a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:14.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.833+0000 7f836c6b1640 1 --2- 172.21.15.131:0/4264097925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f836406fed0 0x7f836415a620 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8354006960 tx=0x7f8354001640 comp rx=0 tx=0).stop 2023-09-12T14:55:14.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.833+0000 7f836c6b1640 1 -- 172.21.15.131:0/4264097925 shutdown_connections 2023-09-12T14:55:14.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.833+0000 7f836c6b1640 1 --2- 172.21.15.131:0/4264097925 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8348065b20 0x7f8348067fe0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:14.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.833+0000 7f836c6b1640 1 --2- 172.21.15.131:0/4264097925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83640cc5f0 0x7f83640cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:14.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.833+0000 7f836c6b1640 1 --2- 172.21.15.131:0/4264097925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f836406fed0 0x7f836415a620 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:14.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.833+0000 7f836c6b1640 1 --2- 172.21.15.131:0/4264097925 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83500039b0 0x7f836409e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:14.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.833+0000 7f836c6b1640 1 -- 172.21.15.131:0/4264097925 >> 172.21.15.131:0/4264097925 conn(0x7f836409a200 msgr2=0x7f83640ca380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:14.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.833+0000 7f836c6b1640 1 -- 172.21.15.131:0/4264097925 shutdown_connections 2023-09-12T14:55:14.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:14.833+0000 7f836c6b1640 1 -- 172.21.15.131:0/4264097925 wait complete. 2023-09-12T14:55:14.854 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-12T14:55:14.854 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-12T14:55:14.855 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2023-09-12T14:55:14.855 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-09-12T14:55:14.855 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-09-12T14:55:14.855 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-12T14:55:14.856 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-12T14:55:14.856 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-12T14:55:14.856 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-09-12T14:55:14.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:14.857 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2023-09-12T14:55:14.863 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-12T14:55:14.863 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-12T14:55:14.863 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2023-09-12T14:55:14.863 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:14.863 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2023-09-12T14:55:14.864 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:14.864 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-12T14:55:14.864 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:55:14.864 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2023-09-12T14:55:14.864 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-12T14:55:14.865 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:55:14.865 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2023-09-12T14:55:14.865 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-12T14:55:14.865 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:55:14.865 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-12T14:55:14.866 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:55:14.866 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-09-12T14:55:14.866 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42481sec preferred_lft 42481sec 2023-09-12T14:55:14.866 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2023-09-12T14:55:14.866 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:14.867 INFO:teuthology.orchestra.run.smithi131.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-09-12T14:55:14.867 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 02:42:21:fb:7e:42 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:55:14.867 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-09-12T14:55:14.867 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:14.867 INFO:teuthology.orchestra.run.smithi131.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-12T14:55:14.867 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-09-12T14:55:14.868 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-12T14:55:14.868 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:14.868 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-09-12T14:55:14.868 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:14.868 INFO:teuthology.orchestra.run.smithi131.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-12T14:55:14.869 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-12T14:55:14.869 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-09-12T14:55:14.869 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:14.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:14.870 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-12T14:55:14.921 INFO:teuthology.orchestra.run.smithi131.stdout:1 2023-09-12T14:55:14.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:14.923 DEBUG:teuthology.orchestra.run.smithi131:> route 2023-09-12T14:55:14.973 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2023-09-12T14:55:14.973 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-12T14:55:14.974 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-09-12T14:55:14.974 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-12T14:55:14.974 INFO:teuthology.orchestra.run.smithi131.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-09-12T14:55:14.974 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-12T14:55:14.974 INFO:teuthology.orchestra.run.smithi131.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-12T14:55:14.975 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-12T14:55:14.975 INFO:teuthology.orchestra.run.smithi131.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-12T14:55:14.975 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-12T14:55:14.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:14.976 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-09-12T14:55:14.976 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-09-12T14:55:14.976 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-12T14:55:14.977 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-12T14:55:14.977 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-12T14:55:14.977 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-09-12T14:55:15.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:15.051 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2023-09-12T14:55:15.098 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-12T14:55:15.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:15.100 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list-id 2023-09-12T14:55:15.146 INFO:teuthology.orchestra.run.smithi131.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-09-12T14:55:15.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:15.147 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-09-12T14:55:15.148 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-09-12T14:55:15.148 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-12T14:55:15.148 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-09-12T14:55:15.148 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-09-12T14:55:15.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:15.219 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-09-12T14:55:15.275 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-12T14:55:15.275 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-12T14:55:15.275 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2023-09-12T14:55:15.276 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:15.276 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2023-09-12T14:55:15.276 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:15.276 INFO:teuthology.orchestra.run.smithi131.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-12T14:55:15.277 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 72:f1:78:22:e9:65 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-09-12T14:55:15.277 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-09-12T14:55:15.277 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:15.277 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::70f1:78ff:fe22:e965/64 scope link 2023-09-12T14:55:15.277 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:15.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:15.279 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-09-12T14:55:15.334 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-12T14:55:15.334 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-12T14:55:15.335 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2023-09-12T14:55:15.335 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:15.335 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2023-09-12T14:55:15.335 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:15.336 INFO:teuthology.orchestra.run.smithi131.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-12T14:55:15.336 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 72:f1:78:22:e9:65 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-09-12T14:55:15.336 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-09-12T14:55:15.336 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:15.336 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::70f1:78ff:fe22:e965/64 scope link 2023-09-12T14:55:15.337 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:15.338 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-09-12T14:55:15.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:15.338 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-09-12T14:55:15.339 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-09-12T14:55:15.339 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-09-12T14:55:15.339 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-09-12T14:55:15.339 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-09-12T14:55:15.340 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-09-12T14:55:15.340 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-09-12T14:55:15.340 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-09-12T14:55:15.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:15.440 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-09-12T14:55:15.441 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-09-12T14:55:15.441 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.1 up 2023-09-12T14:55:15.441 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set dev brx.1 master ceph-brx 2023-09-12T14:55:15.442 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-09-12T14:55:15.462 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-09-12T14:55:15.462 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-12T14:55:15.463 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-09-12T14:55:15.510 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-09-12T14:55:15.510 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-12T14:55:15.511 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-09-12T14:55:15.557 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2023-09-12T14:55:15.616 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-12T14:55:15.616 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-12T14:55:15.669 INFO:teuthology.orchestra.run.smithi131.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-12T14:55:15.670 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-12T14:55:15.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:15.671 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2023-09-12T14:55:15.718 INFO:teuthology.orchestra.run.smithi131.stdout:45 2023-09-12T14:55:15.719 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2023-09-12T14:55:15.719 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-09-12T14:55:15.763 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2023-09-12T14:55:15.778 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-12T14:55:15.778 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-12T14:55:15.830 INFO:teuthology.orchestra.run.smithi131.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-12T14:55:15.831 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-12T14:55:15.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:15.832 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2023-09-12T14:55:15.843 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi131.stderr:2023-09-12T14:55:15.841+0000 7f15f5542580 -1 init, newargv = 0x558ef8024720 newargc=15 2023-09-12T14:55:15.844 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi131.stderr:ceph-fuse[46163]: starting ceph client 2023-09-12T14:55:15.848 INFO:teuthology.orchestra.run.smithi131.stdout:45 2023-09-12T14:55:15.851 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi131.stderr:ceph-fuse[46163]: starting fuse 2023-09-12T14:55:16.950 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2023-09-12T14:55:16.967 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-12T14:55:16.968 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-12T14:55:17.022 INFO:teuthology.orchestra.run.smithi131.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-12T14:55:17.023 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-12T14:55:17.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:17.024 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2023-09-12T14:55:17.071 INFO:teuthology.orchestra.run.smithi131.stdout:45 2023-09-12T14:55:17.071 INFO:teuthology.orchestra.run.smithi131.stdout:46 2023-09-12T14:55:17.072 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2023-09-12T14:55:17.072 DEBUG:teuthology.orchestra.run.smithi131:> sudo stdin-killer -- python3 -c ' 2023-09-12T14:55:17.072 DEBUG:teuthology.orchestra.run.smithi131:> import glob 2023-09-12T14:55:17.073 DEBUG:teuthology.orchestra.run.smithi131:> import re 2023-09-12T14:55:17.073 DEBUG:teuthology.orchestra.run.smithi131:> import os 2023-09-12T14:55:17.074 DEBUG:teuthology.orchestra.run.smithi131:> import subprocess 2023-09-12T14:55:17.074 DEBUG:teuthology.orchestra.run.smithi131:> 2023-09-12T14:55:17.074 DEBUG:teuthology.orchestra.run.smithi131:> def _find_admin_socket(client_name): 2023-09-12T14:55:17.075 DEBUG:teuthology.orchestra.run.smithi131:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-09-12T14:55:17.075 DEBUG:teuthology.orchestra.run.smithi131:> files = glob.glob(asok_path) 2023-09-12T14:55:17.075 DEBUG:teuthology.orchestra.run.smithi131:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-09-12T14:55:17.076 DEBUG:teuthology.orchestra.run.smithi131:> 2023-09-12T14:55:17.077 DEBUG:teuthology.orchestra.run.smithi131:> # Given a non-glob path, it better be there 2023-09-12T14:55:17.077 DEBUG:teuthology.orchestra.run.smithi131:> if "*" not in asok_path: 2023-09-12T14:55:17.077 DEBUG:teuthology.orchestra.run.smithi131:> assert(len(files) == 1) 2023-09-12T14:55:17.077 DEBUG:teuthology.orchestra.run.smithi131:> return files[0] 2023-09-12T14:55:17.078 DEBUG:teuthology.orchestra.run.smithi131:> 2023-09-12T14:55:17.078 DEBUG:teuthology.orchestra.run.smithi131:> for f in files: 2023-09-12T14:55:17.079 DEBUG:teuthology.orchestra.run.smithi131:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-12T14:55:17.079 DEBUG:teuthology.orchestra.run.smithi131:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-12T14:55:17.079 DEBUG:teuthology.orchestra.run.smithi131:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-12T14:55:17.080 DEBUG:teuthology.orchestra.run.smithi131:> contents = proc_f.read() 2023-09-12T14:55:17.080 DEBUG:teuthology.orchestra.run.smithi131:> if mountpoint in contents: 2023-09-12T14:55:17.080 DEBUG:teuthology.orchestra.run.smithi131:> return f 2023-09-12T14:55:17.081 DEBUG:teuthology.orchestra.run.smithi131:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-12T14:55:17.081 DEBUG:teuthology.orchestra.run.smithi131:> 2023-09-12T14:55:17.081 DEBUG:teuthology.orchestra.run.smithi131:> print(_find_admin_socket("client.2")) 2023-09-12T14:55:17.082 DEBUG:teuthology.orchestra.run.smithi131:> ' 2023-09-12T14:55:17.195 INFO:teuthology.orchestra.run.smithi131.stdout:/var/run/ceph/ceph-client.2.46163.asok 2023-09-12T14:55:17.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-12T14:55:17.206 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.46163.asok 2023-09-12T14:55:17.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:17.207 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.46163.asok status 2023-09-12T14:55:17.363 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-09-12T14:55:17.363 INFO:teuthology.orchestra.run.smithi131.stdout: "metadata": { 2023-09-12T14:55:17.364 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_sha1": "2110e00747b31a0d73768c9e5229da18b26b8aa0", 2023-09-12T14:55:17.364 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 18.0.0-6088-g2110e007 (2110e00747b31a0d73768c9e5229da18b26b8aa0) reef (dev)", 2023-09-12T14:55:17.364 INFO:teuthology.orchestra.run.smithi131.stdout: "entity_id": "2", 2023-09-12T14:55:17.364 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": "smithi131", 2023-09-12T14:55:17.365 INFO:teuthology.orchestra.run.smithi131.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-09-12T14:55:17.365 INFO:teuthology.orchestra.run.smithi131.stdout: "pid": "46163", 2023-09-12T14:55:17.365 INFO:teuthology.orchestra.run.smithi131.stdout: "root": "/" 2023-09-12T14:55:17.365 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:17.365 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_count": 0, 2023-09-12T14:55:17.366 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_pinned_count": 0, 2023-09-12T14:55:17.366 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 4731, 2023-09-12T14:55:17.366 INFO:teuthology.orchestra.run.smithi131.stdout: "inst": { 2023-09-12T14:55:17.366 INFO:teuthology.orchestra.run.smithi131.stdout: "name": { 2023-09-12T14:55:17.366 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "client", 2023-09-12T14:55:17.367 INFO:teuthology.orchestra.run.smithi131.stdout: "num": 4731 2023-09-12T14:55:17.367 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:17.367 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2023-09-12T14:55:17.367 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2023-09-12T14:55:17.367 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.2:0", 2023-09-12T14:55:17.367 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 1783137260 2023-09-12T14:55:17.368 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:17.368 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:17.368 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2023-09-12T14:55:17.368 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2023-09-12T14:55:17.368 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.2:0", 2023-09-12T14:55:17.369 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 1783137260 2023-09-12T14:55:17.369 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:17.369 INFO:teuthology.orchestra.run.smithi131.stdout: "inst_str": "client.4731 192.168.0.2:0/1783137260", 2023-09-12T14:55:17.369 INFO:teuthology.orchestra.run.smithi131.stdout: "addr_str": "192.168.0.2:0/1783137260", 2023-09-12T14:55:17.370 INFO:teuthology.orchestra.run.smithi131.stdout: "inode_count": 1, 2023-09-12T14:55:17.370 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_epoch": 6, 2023-09-12T14:55:17.370 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch": 20, 2023-09-12T14:55:17.371 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch_barrier": 0, 2023-09-12T14:55:17.371 INFO:teuthology.orchestra.run.smithi131.stdout: "blocklisted": false, 2023-09-12T14:55:17.371 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs" 2023-09-12T14:55:17.371 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-09-12T14:55:17.377 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-12T14:55:17.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff80d387640 1 -- 172.21.15.131:0/2867595865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff808158180 msgr2=0x7ff8080cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:17.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff80d387640 1 --2- 172.21.15.131:0/2867595865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff808158180 0x7ff8080cae70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff80806eea0 tx=0x7ff7fc01c870 comp rx=0 tx=0).stop 2023-09-12T14:55:17.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff80d387640 1 -- 172.21.15.131:0/2867595865 shutdown_connections 2023-09-12T14:55:17.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff80d387640 1 --2- 172.21.15.131:0/2867595865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff808158180 0x7ff8080cae70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:17.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff80d387640 1 --2- 172.21.15.131:0/2867595865 >> v2:172.21.15.187:3300/0 conn(0x7ff8080ab730 0x7ff8080abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:17.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff80d387640 1 -- 172.21.15.131:0/2867595865 >> 172.21.15.131:0/2867595865 conn(0x7ff808069840 msgr2=0x7ff8080cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:17.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff80d387640 1 -- 172.21.15.131:0/2867595865 shutdown_connections 2023-09-12T14:55:17.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff80d387640 1 -- 172.21.15.131:0/2867595865 wait complete. 2023-09-12T14:55:17.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff80d387640 1 Processor -- start 2023-09-12T14:55:17.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff80d387640 1 -- start start 2023-09-12T14:55:17.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff80d387640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff80806f390 0x7ff8081436d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:17.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff80d387640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8080ab730 0x7ff808147e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:17.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff80d387640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff808158180 0x7ff8080ac470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:17.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff80d387640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80814ddf0 con 0x7ff808158180 2023-09-12T14:55:17.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff80d387640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80815a6d0 con 0x7ff80806f390 2023-09-12T14:55:17.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff80d387640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff80815a110 con 0x7ff8080ab730 2023-09-12T14:55:17.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff8077fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff808158180 0x7ff8080ac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:17.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff8077fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff808158180 0x7ff8080ac470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41552/0 (socket says 172.21.15.131:41552) 2023-09-12T14:55:17.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff8077fe640 1 -- 172.21.15.131:0/3121641134 learned_addr learned my addr 172.21.15.131:0/3121641134 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:17.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff806ffd640 1 --2- 172.21.15.131:0/3121641134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff80806f390 0x7ff8081436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:17.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff8067fc640 1 --2- 172.21.15.131:0/3121641134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8080ab730 0x7ff808147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:17.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff8077fe640 1 -- 172.21.15.131:0/3121641134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8080ab730 msgr2=0x7ff808147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:17.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff8077fe640 1 --2- 172.21.15.131:0/3121641134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8080ab730 0x7ff808147e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:17.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff8077fe640 1 -- 172.21.15.131:0/3121641134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff80806f390 msgr2=0x7ff8081436d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:17.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff8077fe640 1 --2- 172.21.15.131:0/3121641134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff80806f390 0x7ff8081436d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:17.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff8077fe640 1 -- 172.21.15.131:0/3121641134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7fc009560 con 0x7ff808158180 2023-09-12T14:55:17.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff8067fc640 1 --2- 172.21.15.131:0/3121641134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8080ab730 0x7ff808147e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:17.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff8077fe640 1 --2- 172.21.15.131:0/3121641134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff808158180 0x7ff8080ac470 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff800001570 tx=0x7ff80000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:17.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff7ef7fe640 1 -- 172.21.15.131:0/3121641134 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff800003db0 con 0x7ff808158180 2023-09-12T14:55:17.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff7ef7fe640 1 -- 172.21.15.131:0/3121641134 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff800005070 con 0x7ff808158180 2023-09-12T14:55:17.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff80d387640 1 -- 172.21.15.131:0/3121641134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff80815a3f0 con 0x7ff808158180 2023-09-12T14:55:17.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff7ef7fe640 1 -- 172.21.15.131:0/3121641134 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff800005480 con 0x7ff808158180 2023-09-12T14:55:17.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.501+0000 7ff80d387640 1 -- 172.21.15.131:0/3121641134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8080b0510 con 0x7ff808158180 2023-09-12T14:55:17.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.505+0000 7ff80d387640 1 -- 172.21.15.131:0/3121641134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8080b0700 con 0x7ff808158180 2023-09-12T14:55:17.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.505+0000 7ff7ef7fe640 1 -- 172.21.15.131:0/3121641134 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7ff8000055e0 con 0x7ff808158180 2023-09-12T14:55:17.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.505+0000 7ff7ef7fe640 1 --2- 172.21.15.131:0/3121641134 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7ff7f4065b20 0x7ff7f4067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:17.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.505+0000 7ff7ef7fe640 1 -- 172.21.15.131:0/3121641134 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff800082e20 con 0x7ff808158180 2023-09-12T14:55:17.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.505+0000 7ff806ffd640 1 --2- 172.21.15.131:0/3121641134 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7ff7f4065b20 0x7ff7f4067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:17.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.505+0000 7ff806ffd640 1 --2- 172.21.15.131:0/3121641134 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7ff7f4065b20 0x7ff7f4067fe0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff7f80058d0 tx=0x7ff7f8005860 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:17.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.509+0000 7ff7ef7fe640 1 -- 172.21.15.131:0/3121641134 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7ff800050760 con 0x7ff808158180 2023-09-12T14:55:17.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.697+0000 7ff80d387640 1 -- 172.21.15.131:0/3121641134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff808073580 con 0x7ff808158180 2023-09-12T14:55:17.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.697+0000 7ff7ef7fe640 1 -- 172.21.15.131:0/3121641134 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff800017020 con 0x7ff808158180 2023-09-12T14:55:17.700 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-12T14:55:17.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.701+0000 7ff80d387640 1 -- 172.21.15.131:0/3121641134 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7ff7f4065b20 msgr2=0x7ff7f4067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:17.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.701+0000 7ff80d387640 1 --2- 172.21.15.131:0/3121641134 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7ff7f4065b20 0x7ff7f4067fe0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff7f80058d0 tx=0x7ff7f8005860 comp rx=0 tx=0).stop 2023-09-12T14:55:17.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.701+0000 7ff80d387640 1 -- 172.21.15.131:0/3121641134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff808158180 msgr2=0x7ff8080ac470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:17.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.701+0000 7ff80d387640 1 --2- 172.21.15.131:0/3121641134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff808158180 0x7ff8080ac470 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff800001570 tx=0x7ff80000cd20 comp rx=0 tx=0).stop 2023-09-12T14:55:17.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.701+0000 7ff80d387640 1 -- 172.21.15.131:0/3121641134 shutdown_connections 2023-09-12T14:55:17.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.701+0000 7ff80d387640 1 --2- 172.21.15.131:0/3121641134 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7ff7f4065b20 0x7ff7f4067fe0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:17.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.701+0000 7ff80d387640 1 --2- 172.21.15.131:0/3121641134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8080ab730 0x7ff808147e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:17.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.701+0000 7ff80d387640 1 --2- 172.21.15.131:0/3121641134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff808158180 0x7ff8080ac470 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:17.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.701+0000 7ff80d387640 1 --2- 172.21.15.131:0/3121641134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff80806f390 0x7ff8081436d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:17.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.701+0000 7ff80d387640 1 -- 172.21.15.131:0/3121641134 >> 172.21.15.131:0/3121641134 conn(0x7ff808069840 msgr2=0x7ff808149620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:17.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.701+0000 7ff80d387640 1 -- 172.21.15.131:0/3121641134 shutdown_connections 2023-09-12T14:55:17.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:17.701+0000 7ff80d387640 1 -- 172.21.15.131:0/3121641134 wait complete. 2023-09-12T14:55:17.719 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-12T14:55:17.719 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-12T14:55:17.719 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi187.front.sepia.ceph.com 2023-09-12T14:55:17.720 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-09-12T14:55:17.720 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-09-12T14:55:17.720 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-12T14:55:17.720 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-12T14:55:17.721 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-12T14:55:17.721 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-09-12T14:55:17.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:17.722 DEBUG:teuthology.orchestra.run.smithi187:> ip addr 2023-09-12T14:55:17.729 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-12T14:55:17.729 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-12T14:55:17.729 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2023-09-12T14:55:17.729 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:17.730 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2023-09-12T14:55:17.730 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:17.730 INFO:teuthology.orchestra.run.smithi187.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-12T14:55:17.730 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:78 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:55:17.731 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f0 2023-09-12T14:55:17.731 INFO:teuthology.orchestra.run.smithi187.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-12T14:55:17.731 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:79 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:55:17.731 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f1 2023-09-12T14:55:17.731 INFO:teuthology.orchestra.run.smithi187.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-12T14:55:17.732 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b0 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:55:17.732 INFO:teuthology.orchestra.run.smithi187.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-12T14:55:17.732 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b1 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:55:17.732 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.21.15.187/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-09-12T14:55:17.732 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft 42490sec preferred_lft 42490sec 2023-09-12T14:55:17.732 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ec4:7aff:fe88:77b1/64 scope link 2023-09-12T14:55:17.733 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:17.733 INFO:teuthology.orchestra.run.smithi187.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-09-12T14:55:17.733 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 02:42:cb:e2:88:30 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:55:17.733 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-09-12T14:55:17.733 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:17.734 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-09-12T14:55:17.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:17.735 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec bash -c ' 2023-09-12T14:55:17.735 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2023-09-12T14:55:17.735 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link add name ceph-brx type bridge 2023-09-12T14:55:17.735 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip addr flush dev ceph-brx 2023-09-12T14:55:17.736 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set ceph-brx up 2023-09-12T14:55:17.736 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-09-12T14:55:17.736 DEBUG:teuthology.orchestra.run.smithi187:> ') 2023-09-12T14:55:17.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:17.816 DEBUG:teuthology.orchestra.run.smithi187:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-12T14:55:17.868 INFO:teuthology.orchestra.run.smithi187.stdout:1 2023-09-12T14:55:17.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:17.869 DEBUG:teuthology.orchestra.run.smithi187:> route 2023-09-12T14:55:18.028 INFO:teuthology.orchestra.run.smithi187.stdout:Kernel IP routing table 2023-09-12T14:55:18.028 INFO:teuthology.orchestra.run.smithi187.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-12T14:55:18.028 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-09-12T14:55:18.028 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-12T14:55:18.029 INFO:teuthology.orchestra.run.smithi187.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-09-12T14:55:18.029 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-12T14:55:18.029 INFO:teuthology.orchestra.run.smithi187.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-12T14:55:18.029 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-12T14:55:18.029 INFO:teuthology.orchestra.run.smithi187.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-12T14:55:18.030 INFO:teuthology.orchestra.run.smithi187.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-12T14:55:18.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:18.030 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec bash -c ' 2023-09-12T14:55:18.031 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2023-09-12T14:55:18.031 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-12T14:55:18.031 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-12T14:55:18.032 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-12T14:55:18.032 DEBUG:teuthology.orchestra.run.smithi187:> ') 2023-09-12T14:55:18.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:18.070 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2023-09-12T14:55:18.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:18.116 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list-id 2023-09-12T14:55:18.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:18.164 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec bash -c ' 2023-09-12T14:55:18.164 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2023-09-12T14:55:18.164 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-09-12T14:55:18.165 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-09-12T14:55:18.165 DEBUG:teuthology.orchestra.run.smithi187:> ') 2023-09-12T14:55:18.231 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-09-12T14:55:18.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:18.231 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec bash -c ' 2023-09-12T14:55:18.232 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2023-09-12T14:55:18.232 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-09-12T14:55:18.232 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-09-12T14:55:18.233 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-09-12T14:55:18.233 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-09-12T14:55:18.233 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-09-12T14:55:18.233 DEBUG:teuthology.orchestra.run.smithi187:> ') 2023-09-12T14:55:18.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:18.347 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec bash -c ' 2023-09-12T14:55:18.347 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2023-09-12T14:55:18.348 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set brx.0 up 2023-09-12T14:55:18.348 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set dev brx.0 master ceph-brx 2023-09-12T14:55:18.348 DEBUG:teuthology.orchestra.run.smithi187:> ') 2023-09-12T14:55:18.368 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-09-12T14:55:18.369 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-12T14:55:18.369 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-09-12T14:55:18.414 INFO:teuthology.orchestra.run.smithi187.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-09-12T14:55:18.415 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-12T14:55:18.415 DEBUG:teuthology.orchestra.run.smithi187:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-09-12T14:55:18.462 DEBUG:teuthology.orchestra.run.smithi187:> sudo modprobe fuse 2023-09-12T14:55:18.517 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-12T14:55:18.518 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-12T14:55:18.568 INFO:teuthology.orchestra.run.smithi187.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-12T14:55:18.569 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-12T14:55:18.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:18.570 DEBUG:teuthology.orchestra.run.smithi187:> ls /sys/fs/fuse/connections 2023-09-12T14:55:18.615 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-12T14:55:18.615 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-09-12T14:55:18.660 DEBUG:teuthology.orchestra.run.smithi187:> sudo modprobe fuse 2023-09-12T14:55:18.672 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-12T14:55:18.672 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-12T14:55:18.725 INFO:teuthology.orchestra.run.smithi187.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-12T14:55:18.726 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-12T14:55:18.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:18.727 DEBUG:teuthology.orchestra.run.smithi187:> ls /sys/fs/fuse/connections 2023-09-12T14:55:18.740 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi187.stderr:2023-09-12T14:55:18.738+0000 7f8eb9e66580 -1 init, newargv = 0x559b8bf547b0 newargc=15 2023-09-12T14:55:18.741 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi187.stderr:ceph-fuse[43886]: starting ceph client 2023-09-12T14:55:18.747 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi187.stderr:ceph-fuse[43886]: starting fuse 2023-09-12T14:55:19.846 DEBUG:teuthology.orchestra.run.smithi187:> sudo modprobe fuse 2023-09-12T14:55:19.865 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-12T14:55:19.866 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-12T14:55:19.917 INFO:teuthology.orchestra.run.smithi187.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-12T14:55:19.918 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-12T14:55:19.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:19.919 DEBUG:teuthology.orchestra.run.smithi187:> ls /sys/fs/fuse/connections 2023-09-12T14:55:19.967 INFO:teuthology.orchestra.run.smithi187.stdout:45 2023-09-12T14:55:19.967 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-09-12T14:55:19.968 DEBUG:teuthology.orchestra.run.smithi187:> sudo stdin-killer -- python3 -c ' 2023-09-12T14:55:19.968 DEBUG:teuthology.orchestra.run.smithi187:> import glob 2023-09-12T14:55:19.968 DEBUG:teuthology.orchestra.run.smithi187:> import re 2023-09-12T14:55:19.969 DEBUG:teuthology.orchestra.run.smithi187:> import os 2023-09-12T14:55:19.969 DEBUG:teuthology.orchestra.run.smithi187:> import subprocess 2023-09-12T14:55:19.969 DEBUG:teuthology.orchestra.run.smithi187:> 2023-09-12T14:55:19.970 DEBUG:teuthology.orchestra.run.smithi187:> def _find_admin_socket(client_name): 2023-09-12T14:55:19.970 DEBUG:teuthology.orchestra.run.smithi187:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-09-12T14:55:19.970 DEBUG:teuthology.orchestra.run.smithi187:> files = glob.glob(asok_path) 2023-09-12T14:55:19.971 DEBUG:teuthology.orchestra.run.smithi187:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-09-12T14:55:19.971 DEBUG:teuthology.orchestra.run.smithi187:> 2023-09-12T14:55:19.971 DEBUG:teuthology.orchestra.run.smithi187:> # Given a non-glob path, it better be there 2023-09-12T14:55:19.971 DEBUG:teuthology.orchestra.run.smithi187:> if "*" not in asok_path: 2023-09-12T14:55:19.972 DEBUG:teuthology.orchestra.run.smithi187:> assert(len(files) == 1) 2023-09-12T14:55:19.972 DEBUG:teuthology.orchestra.run.smithi187:> return files[0] 2023-09-12T14:55:19.972 DEBUG:teuthology.orchestra.run.smithi187:> 2023-09-12T14:55:19.973 DEBUG:teuthology.orchestra.run.smithi187:> for f in files: 2023-09-12T14:55:19.973 DEBUG:teuthology.orchestra.run.smithi187:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-12T14:55:19.973 DEBUG:teuthology.orchestra.run.smithi187:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-12T14:55:19.974 DEBUG:teuthology.orchestra.run.smithi187:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-12T14:55:19.974 DEBUG:teuthology.orchestra.run.smithi187:> contents = proc_f.read() 2023-09-12T14:55:19.974 DEBUG:teuthology.orchestra.run.smithi187:> if mountpoint in contents: 2023-09-12T14:55:19.974 DEBUG:teuthology.orchestra.run.smithi187:> return f 2023-09-12T14:55:19.975 DEBUG:teuthology.orchestra.run.smithi187:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-12T14:55:19.975 DEBUG:teuthology.orchestra.run.smithi187:> 2023-09-12T14:55:19.975 DEBUG:teuthology.orchestra.run.smithi187:> print(_find_admin_socket("client.1")) 2023-09-12T14:55:19.976 DEBUG:teuthology.orchestra.run.smithi187:> ' 2023-09-12T14:55:20.091 INFO:teuthology.orchestra.run.smithi187.stdout:/var/run/ceph/ceph-client.1.43886.asok 2023-09-12T14:55:20.095 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:55:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-12T14:55:20.103 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.43886.asok 2023-09-12T14:55:20.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:20.104 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.43886.asok status 2023-09-12T14:55:20.257 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2023-09-12T14:55:20.257 INFO:teuthology.orchestra.run.smithi187.stdout: "metadata": { 2023-09-12T14:55:20.258 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_sha1": "2110e00747b31a0d73768c9e5229da18b26b8aa0", 2023-09-12T14:55:20.258 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_version": "ceph version 18.0.0-6088-g2110e007 (2110e00747b31a0d73768c9e5229da18b26b8aa0) reef (dev)", 2023-09-12T14:55:20.258 INFO:teuthology.orchestra.run.smithi187.stdout: "entity_id": "1", 2023-09-12T14:55:20.258 INFO:teuthology.orchestra.run.smithi187.stdout: "hostname": "smithi187", 2023-09-12T14:55:20.259 INFO:teuthology.orchestra.run.smithi187.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-09-12T14:55:20.259 INFO:teuthology.orchestra.run.smithi187.stdout: "pid": "43886", 2023-09-12T14:55:20.259 INFO:teuthology.orchestra.run.smithi187.stdout: "root": "/" 2023-09-12T14:55:20.259 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-09-12T14:55:20.259 INFO:teuthology.orchestra.run.smithi187.stdout: "dentry_count": 0, 2023-09-12T14:55:20.260 INFO:teuthology.orchestra.run.smithi187.stdout: "dentry_pinned_count": 0, 2023-09-12T14:55:20.260 INFO:teuthology.orchestra.run.smithi187.stdout: "id": 4669, 2023-09-12T14:55:20.260 INFO:teuthology.orchestra.run.smithi187.stdout: "inst": { 2023-09-12T14:55:20.260 INFO:teuthology.orchestra.run.smithi187.stdout: "name": { 2023-09-12T14:55:20.260 INFO:teuthology.orchestra.run.smithi187.stdout: "type": "client", 2023-09-12T14:55:20.261 INFO:teuthology.orchestra.run.smithi187.stdout: "num": 4669 2023-09-12T14:55:20.261 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-09-12T14:55:20.261 INFO:teuthology.orchestra.run.smithi187.stdout: "addr": { 2023-09-12T14:55:20.261 INFO:teuthology.orchestra.run.smithi187.stdout: "type": "v1", 2023-09-12T14:55:20.262 INFO:teuthology.orchestra.run.smithi187.stdout: "addr": "192.168.0.1:0", 2023-09-12T14:55:20.262 INFO:teuthology.orchestra.run.smithi187.stdout: "nonce": 824248995 2023-09-12T14:55:20.262 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-09-12T14:55:20.262 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-09-12T14:55:20.262 INFO:teuthology.orchestra.run.smithi187.stdout: "addr": { 2023-09-12T14:55:20.263 INFO:teuthology.orchestra.run.smithi187.stdout: "type": "v1", 2023-09-12T14:55:20.263 INFO:teuthology.orchestra.run.smithi187.stdout: "addr": "192.168.0.1:0", 2023-09-12T14:55:20.263 INFO:teuthology.orchestra.run.smithi187.stdout: "nonce": 824248995 2023-09-12T14:55:20.263 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-09-12T14:55:20.264 INFO:teuthology.orchestra.run.smithi187.stdout: "inst_str": "client.4669 192.168.0.1:0/824248995", 2023-09-12T14:55:20.264 INFO:teuthology.orchestra.run.smithi187.stdout: "addr_str": "192.168.0.1:0/824248995", 2023-09-12T14:55:20.264 INFO:teuthology.orchestra.run.smithi187.stdout: "inode_count": 1, 2023-09-12T14:55:20.264 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_epoch": 6, 2023-09-12T14:55:20.264 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_epoch": 20, 2023-09-12T14:55:20.265 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_epoch_barrier": 0, 2023-09-12T14:55:20.265 INFO:teuthology.orchestra.run.smithi187.stdout: "blocklisted": false, 2023-09-12T14:55:20.265 INFO:teuthology.orchestra.run.smithi187.stdout: "fs_name": "cephfs" 2023-09-12T14:55:20.265 INFO:teuthology.orchestra.run.smithi187.stdout:} 2023-09-12T14:55:20.272 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-12T14:55:20.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f759386f640 1 -- 172.21.15.131:0/1575291195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75780047d0 msgr2=0x7f7578004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:20.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f759386f640 1 --2- 172.21.15.131:0/1575291195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75780047d0 0x7f7578004c50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f758c065700 tx=0x7f757c01cf50 comp rx=0 tx=0).stop 2023-09-12T14:55:20.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f759386f640 1 -- 172.21.15.131:0/1575291195 shutdown_connections 2023-09-12T14:55:20.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f759386f640 1 --2- 172.21.15.131:0/1575291195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75780047d0 0x7f7578004c50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:20.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f759386f640 1 -- 172.21.15.131:0/1575291195 >> 172.21.15.131:0/1575291195 conn(0x7f758c0c9b10 msgr2=0x7f758c0c9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:20.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f759386f640 1 -- 172.21.15.131:0/1575291195 shutdown_connections 2023-09-12T14:55:20.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f759386f640 1 -- 172.21.15.131:0/1575291195 wait complete. 2023-09-12T14:55:20.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f759386f640 1 Processor -- start 2023-09-12T14:55:20.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f759386f640 1 -- start start 2023-09-12T14:55:20.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f759386f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75780047d0 0x7f758c0b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:20.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f759386f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f758c15a440 0x7f758c1f4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:20.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f759386f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f758c159830 0x7f758c159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:20.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f759386f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c15a1f0 con 0x7f75780047d0 2023-09-12T14:55:20.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f759386f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c0cc700 con 0x7f758c159830 2023-09-12T14:55:20.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f759386f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758c0cb6a0 con 0x7f758c15a440 2023-09-12T14:55:20.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f759160b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75780047d0 0x7f758c0b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:20.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f759160b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75780047d0 0x7f758c0b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55830/0 (socket says 172.21.15.131:55830) 2023-09-12T14:55:20.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f759160b640 1 -- 172.21.15.131:0/3516108000 learned_addr learned my addr 172.21.15.131:0/3516108000 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:20.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f7590e0a640 1 --2- 172.21.15.131:0/3516108000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f758c15a440 0x7f758c1f4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:20.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f7591e0c640 1 --2- 172.21.15.131:0/3516108000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f758c159830 0x7f758c159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:20.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f759160b640 1 -- 172.21.15.131:0/3516108000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f758c15a440 msgr2=0x7f758c1f4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:20.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f759160b640 1 --2- 172.21.15.131:0/3516108000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f758c15a440 0x7f758c1f4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:20.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f759160b640 1 -- 172.21.15.131:0/3516108000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f758c159830 msgr2=0x7f758c159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:20.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f759160b640 1 --2- 172.21.15.131:0/3516108000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f758c159830 0x7f758c159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:20.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f759160b640 1 -- 172.21.15.131:0/3516108000 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758c0cb940 con 0x7f75780047d0 2023-09-12T14:55:20.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f7591e0c640 1 --2- 172.21.15.131:0/3516108000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f758c159830 0x7f758c159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:20.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f759160b640 1 --2- 172.21.15.131:0/3516108000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75780047d0 0x7f758c0b0d10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7584009750 tx=0x7f7584009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:20.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f7581ffb640 1 -- 172.21.15.131:0/3516108000 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758400ece0 con 0x7f75780047d0 2023-09-12T14:55:20.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f7581ffb640 1 -- 172.21.15.131:0/3516108000 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7584009e60 con 0x7f75780047d0 2023-09-12T14:55:20.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f7581ffb640 1 -- 172.21.15.131:0/3516108000 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7584003bc0 con 0x7f75780047d0 2023-09-12T14:55:20.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f759386f640 1 -- 172.21.15.131:0/3516108000 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f758c0cbc20 con 0x7f75780047d0 2023-09-12T14:55:20.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.405+0000 7f759386f640 1 -- 172.21.15.131:0/3516108000 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758c0cc1e0 con 0x7f75780047d0 2023-09-12T14:55:20.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.409+0000 7f759386f640 1 -- 172.21.15.131:0/3516108000 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f758c073580 con 0x7f75780047d0 2023-09-12T14:55:20.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.409+0000 7f7581ffb640 1 -- 172.21.15.131:0/3516108000 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f7584003d20 con 0x7f75780047d0 2023-09-12T14:55:20.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.409+0000 7f7581ffb640 1 --2- 172.21.15.131:0/3516108000 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7570065b20 0x7f7570067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:20.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.409+0000 7f7581ffb640 1 -- 172.21.15.131:0/3516108000 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7584083ec0 con 0x7f75780047d0 2023-09-12T14:55:20.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.409+0000 7f7590e0a640 1 --2- 172.21.15.131:0/3516108000 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7570065b20 0x7f7570067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:20.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.409+0000 7f7590e0a640 1 --2- 172.21.15.131:0/3516108000 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7570065b20 0x7f7570067fe0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f757c0ac000 tx=0x7f757c0b1040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:20.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.413+0000 7f7581ffb640 1 -- 172.21.15.131:0/3516108000 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f75840516f0 con 0x7f75780047d0 2023-09-12T14:55:20.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.601+0000 7f759386f640 1 -- 172.21.15.131:0/3516108000 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f758c0ca870 con 0x7f75780047d0 2023-09-12T14:55:20.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.601+0000 7f7581ffb640 1 -- 172.21.15.131:0/3516108000 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f758401f060 con 0x7f75780047d0 2023-09-12T14:55:20.603 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-12T14:55:20.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.601+0000 7f759386f640 1 -- 172.21.15.131:0/3516108000 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7570065b20 msgr2=0x7f7570067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:20.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.601+0000 7f759386f640 1 --2- 172.21.15.131:0/3516108000 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7570065b20 0x7f7570067fe0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f757c0ac000 tx=0x7f757c0b1040 comp rx=0 tx=0).stop 2023-09-12T14:55:20.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.601+0000 7f759386f640 1 -- 172.21.15.131:0/3516108000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75780047d0 msgr2=0x7f758c0b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:20.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.601+0000 7f759386f640 1 --2- 172.21.15.131:0/3516108000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75780047d0 0x7f758c0b0d10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7584009750 tx=0x7f7584009c10 comp rx=0 tx=0).stop 2023-09-12T14:55:20.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.601+0000 7f759386f640 1 -- 172.21.15.131:0/3516108000 shutdown_connections 2023-09-12T14:55:20.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.601+0000 7f759386f640 1 --2- 172.21.15.131:0/3516108000 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7570065b20 0x7f7570067fe0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:20.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.601+0000 7f759386f640 1 --2- 172.21.15.131:0/3516108000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f758c15a440 0x7f758c1f4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:20.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.601+0000 7f759386f640 1 --2- 172.21.15.131:0/3516108000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f758c159830 0x7f758c159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:20.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.601+0000 7f759386f640 1 --2- 172.21.15.131:0/3516108000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75780047d0 0x7f758c0b0d10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:20.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.601+0000 7f759386f640 1 -- 172.21.15.131:0/3516108000 >> 172.21.15.131:0/3516108000 conn(0x7f758c0c9b10 msgr2=0x7f758c157ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:20.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.601+0000 7f759386f640 1 -- 172.21.15.131:0/3516108000 shutdown_connections 2023-09-12T14:55:20.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:20.605+0000 7f759386f640 1 -- 172.21.15.131:0/3516108000 wait complete. 2023-09-12T14:55:20.623 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-12T14:55:20.623 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-12T14:55:20.623 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi187.front.sepia.ceph.com 2023-09-12T14:55:20.623 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-09-12T14:55:20.624 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-09-12T14:55:20.624 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-12T14:55:20.624 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-12T14:55:20.624 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-12T14:55:20.625 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-09-12T14:55:20.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:20.625 DEBUG:teuthology.orchestra.run.smithi187:> ip addr 2023-09-12T14:55:20.632 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-12T14:55:20.632 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-12T14:55:20.633 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2023-09-12T14:55:20.633 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:20.633 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2023-09-12T14:55:20.633 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:20.633 INFO:teuthology.orchestra.run.smithi187.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-12T14:55:20.634 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:78 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:55:20.634 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f0 2023-09-12T14:55:20.634 INFO:teuthology.orchestra.run.smithi187.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-12T14:55:20.634 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:79 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:55:20.634 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f1 2023-09-12T14:55:20.635 INFO:teuthology.orchestra.run.smithi187.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-12T14:55:20.635 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b0 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:55:20.635 INFO:teuthology.orchestra.run.smithi187.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-12T14:55:20.635 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b1 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:55:20.635 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.21.15.187/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-09-12T14:55:20.635 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft 42487sec preferred_lft 42487sec 2023-09-12T14:55:20.636 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ec4:7aff:fe88:77b1/64 scope link 2023-09-12T14:55:20.636 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:20.636 INFO:teuthology.orchestra.run.smithi187.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-09-12T14:55:20.636 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 02:42:cb:e2:88:30 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:55:20.636 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-09-12T14:55:20.637 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:20.637 INFO:teuthology.orchestra.run.smithi187.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-12T14:55:20.637 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-09-12T14:55:20.637 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-12T14:55:20.637 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:20.637 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-09-12T14:55:20.638 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:20.638 INFO:teuthology.orchestra.run.smithi187.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-12T14:55:20.638 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-09-12T14:55:20.638 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-09-12T14:55:20.638 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:20.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:20.639 DEBUG:teuthology.orchestra.run.smithi187:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-12T14:55:20.686 INFO:teuthology.orchestra.run.smithi187.stdout:1 2023-09-12T14:55:20.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:20.688 DEBUG:teuthology.orchestra.run.smithi187:> route 2023-09-12T14:55:20.738 INFO:teuthology.orchestra.run.smithi187.stdout:Kernel IP routing table 2023-09-12T14:55:20.738 INFO:teuthology.orchestra.run.smithi187.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-12T14:55:20.738 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-09-12T14:55:20.738 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-12T14:55:20.739 INFO:teuthology.orchestra.run.smithi187.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-09-12T14:55:20.739 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-12T14:55:20.739 INFO:teuthology.orchestra.run.smithi187.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-12T14:55:20.739 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-12T14:55:20.740 INFO:teuthology.orchestra.run.smithi187.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-12T14:55:20.740 INFO:teuthology.orchestra.run.smithi187.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-12T14:55:20.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:20.741 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec bash -c ' 2023-09-12T14:55:20.741 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2023-09-12T14:55:20.742 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-12T14:55:20.742 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-12T14:55:20.742 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-12T14:55:20.743 DEBUG:teuthology.orchestra.run.smithi187:> ') 2023-09-12T14:55:20.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:20.815 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2023-09-12T14:55:20.863 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-09-12T14:55:20.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:20.864 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list-id 2023-09-12T14:55:20.911 INFO:teuthology.orchestra.run.smithi187.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-09-12T14:55:20.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:20.912 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec bash -c ' 2023-09-12T14:55:20.912 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2023-09-12T14:55:20.912 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-09-12T14:55:20.913 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-09-12T14:55:20.913 DEBUG:teuthology.orchestra.run.smithi187:> ') 2023-09-12T14:55:20.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:20.986 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-09-12T14:55:21.038 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-12T14:55:21.038 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-12T14:55:21.038 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2023-09-12T14:55:21.039 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:21.039 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2023-09-12T14:55:21.039 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:21.039 INFO:teuthology.orchestra.run.smithi187.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-12T14:55:21.039 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 72:c6:aa:20:3e:97 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-09-12T14:55:21.040 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-09-12T14:55:21.040 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:21.040 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::70c6:aaff:fe20:3e97/64 scope link 2023-09-12T14:55:21.040 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:21.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:21.042 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-09-12T14:55:21.094 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-12T14:55:21.095 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-12T14:55:21.095 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2023-09-12T14:55:21.095 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:21.095 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2023-09-12T14:55:21.095 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:21.096 INFO:teuthology.orchestra.run.smithi187.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-12T14:55:21.096 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 72:c6:aa:20:3e:97 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-09-12T14:55:21.096 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-09-12T14:55:21.096 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:21.096 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::70c6:aaff:fe20:3e97/64 scope link 2023-09-12T14:55:21.097 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:55:21.098 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-09-12T14:55:21.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:21.098 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec bash -c ' 2023-09-12T14:55:21.099 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2023-09-12T14:55:21.099 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-09-12T14:55:21.101 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-09-12T14:55:21.102 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-09-12T14:55:21.102 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-09-12T14:55:21.102 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-09-12T14:55:21.102 DEBUG:teuthology.orchestra.run.smithi187:> ') 2023-09-12T14:55:21.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:21.191 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec bash -c ' 2023-09-12T14:55:21.191 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2023-09-12T14:55:21.191 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set brx.1 up 2023-09-12T14:55:21.192 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set dev brx.1 master ceph-brx 2023-09-12T14:55:21.192 DEBUG:teuthology.orchestra.run.smithi187:> ') 2023-09-12T14:55:21.213 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-09-12T14:55:21.213 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-12T14:55:21.213 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-09-12T14:55:21.258 INFO:teuthology.orchestra.run.smithi187.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-09-12T14:55:21.259 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-12T14:55:21.259 DEBUG:teuthology.orchestra.run.smithi187:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-09-12T14:55:21.307 DEBUG:teuthology.orchestra.run.smithi187:> sudo modprobe fuse 2023-09-12T14:55:21.364 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-12T14:55:21.364 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-12T14:55:21.417 INFO:teuthology.orchestra.run.smithi187.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-12T14:55:21.418 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-12T14:55:21.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:21.419 DEBUG:teuthology.orchestra.run.smithi187:> ls /sys/fs/fuse/connections 2023-09-12T14:55:21.467 INFO:teuthology.orchestra.run.smithi187.stdout:45 2023-09-12T14:55:21.467 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2023-09-12T14:55:21.468 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-09-12T14:55:21.512 DEBUG:teuthology.orchestra.run.smithi187:> sudo modprobe fuse 2023-09-12T14:55:21.528 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-12T14:55:21.528 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-12T14:55:21.584 INFO:teuthology.orchestra.run.smithi187.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-12T14:55:21.585 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-12T14:55:21.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:21.586 DEBUG:teuthology.orchestra.run.smithi187:> ls /sys/fs/fuse/connections 2023-09-12T14:55:21.592 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi187.stderr:2023-09-12T14:55:21.590+0000 7f92930f6580 -1 init, newargv = 0x5630731656a0 newargc=15 2023-09-12T14:55:21.592 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi187.stderr:ceph-fuse[43998]: starting ceph client 2023-09-12T14:55:21.596 INFO:teuthology.orchestra.run.smithi187.stdout:45 2023-09-12T14:55:21.597 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi187.stderr:ceph-fuse[43998]: starting fuse 2023-09-12T14:55:22.698 DEBUG:teuthology.orchestra.run.smithi187:> sudo modprobe fuse 2023-09-12T14:55:22.716 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-12T14:55:22.716 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-12T14:55:22.773 INFO:teuthology.orchestra.run.smithi187.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-12T14:55:22.774 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-12T14:55:22.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:22.776 DEBUG:teuthology.orchestra.run.smithi187:> ls /sys/fs/fuse/connections 2023-09-12T14:55:22.823 INFO:teuthology.orchestra.run.smithi187.stdout:45 2023-09-12T14:55:22.823 INFO:teuthology.orchestra.run.smithi187.stdout:46 2023-09-12T14:55:22.824 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2023-09-12T14:55:22.824 DEBUG:teuthology.orchestra.run.smithi187:> sudo stdin-killer -- python3 -c ' 2023-09-12T14:55:22.824 DEBUG:teuthology.orchestra.run.smithi187:> import glob 2023-09-12T14:55:22.825 DEBUG:teuthology.orchestra.run.smithi187:> import re 2023-09-12T14:55:22.825 DEBUG:teuthology.orchestra.run.smithi187:> import os 2023-09-12T14:55:22.825 DEBUG:teuthology.orchestra.run.smithi187:> import subprocess 2023-09-12T14:55:22.826 DEBUG:teuthology.orchestra.run.smithi187:> 2023-09-12T14:55:22.826 DEBUG:teuthology.orchestra.run.smithi187:> def _find_admin_socket(client_name): 2023-09-12T14:55:22.826 DEBUG:teuthology.orchestra.run.smithi187:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-09-12T14:55:22.827 DEBUG:teuthology.orchestra.run.smithi187:> files = glob.glob(asok_path) 2023-09-12T14:55:22.827 DEBUG:teuthology.orchestra.run.smithi187:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-09-12T14:55:22.827 DEBUG:teuthology.orchestra.run.smithi187:> 2023-09-12T14:55:22.828 DEBUG:teuthology.orchestra.run.smithi187:> # Given a non-glob path, it better be there 2023-09-12T14:55:22.828 DEBUG:teuthology.orchestra.run.smithi187:> if "*" not in asok_path: 2023-09-12T14:55:22.828 DEBUG:teuthology.orchestra.run.smithi187:> assert(len(files) == 1) 2023-09-12T14:55:22.829 DEBUG:teuthology.orchestra.run.smithi187:> return files[0] 2023-09-12T14:55:22.829 DEBUG:teuthology.orchestra.run.smithi187:> 2023-09-12T14:55:22.829 DEBUG:teuthology.orchestra.run.smithi187:> for f in files: 2023-09-12T14:55:22.830 DEBUG:teuthology.orchestra.run.smithi187:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-12T14:55:22.830 DEBUG:teuthology.orchestra.run.smithi187:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-12T14:55:22.830 DEBUG:teuthology.orchestra.run.smithi187:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-12T14:55:22.831 DEBUG:teuthology.orchestra.run.smithi187:> contents = proc_f.read() 2023-09-12T14:55:22.831 DEBUG:teuthology.orchestra.run.smithi187:> if mountpoint in contents: 2023-09-12T14:55:22.831 DEBUG:teuthology.orchestra.run.smithi187:> return f 2023-09-12T14:55:22.831 DEBUG:teuthology.orchestra.run.smithi187:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-12T14:55:22.832 DEBUG:teuthology.orchestra.run.smithi187:> 2023-09-12T14:55:22.832 DEBUG:teuthology.orchestra.run.smithi187:> print(_find_admin_socket("client.3")) 2023-09-12T14:55:22.832 DEBUG:teuthology.orchestra.run.smithi187:> ' 2023-09-12T14:55:22.946 INFO:teuthology.orchestra.run.smithi187.stdout:/var/run/ceph/ceph-client.3.43998.asok 2023-09-12T14:55:22.949 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:55:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-12T14:55:22.958 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.43998.asok 2023-09-12T14:55:22.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:22.959 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.43998.asok status 2023-09-12T14:55:23.111 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2023-09-12T14:55:23.111 INFO:teuthology.orchestra.run.smithi187.stdout: "metadata": { 2023-09-12T14:55:23.111 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_sha1": "2110e00747b31a0d73768c9e5229da18b26b8aa0", 2023-09-12T14:55:23.112 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_version": "ceph version 18.0.0-6088-g2110e007 (2110e00747b31a0d73768c9e5229da18b26b8aa0) reef (dev)", 2023-09-12T14:55:23.112 INFO:teuthology.orchestra.run.smithi187.stdout: "entity_id": "3", 2023-09-12T14:55:23.112 INFO:teuthology.orchestra.run.smithi187.stdout: "hostname": "smithi187", 2023-09-12T14:55:23.113 INFO:teuthology.orchestra.run.smithi187.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-09-12T14:55:23.113 INFO:teuthology.orchestra.run.smithi187.stdout: "pid": "43998", 2023-09-12T14:55:23.113 INFO:teuthology.orchestra.run.smithi187.stdout: "root": "/" 2023-09-12T14:55:23.113 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-09-12T14:55:23.113 INFO:teuthology.orchestra.run.smithi187.stdout: "dentry_count": 0, 2023-09-12T14:55:23.114 INFO:teuthology.orchestra.run.smithi187.stdout: "dentry_pinned_count": 0, 2023-09-12T14:55:23.114 INFO:teuthology.orchestra.run.smithi187.stdout: "id": 4499, 2023-09-12T14:55:23.114 INFO:teuthology.orchestra.run.smithi187.stdout: "inst": { 2023-09-12T14:55:23.114 INFO:teuthology.orchestra.run.smithi187.stdout: "name": { 2023-09-12T14:55:23.114 INFO:teuthology.orchestra.run.smithi187.stdout: "type": "client", 2023-09-12T14:55:23.115 INFO:teuthology.orchestra.run.smithi187.stdout: "num": 4499 2023-09-12T14:55:23.115 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-09-12T14:55:23.115 INFO:teuthology.orchestra.run.smithi187.stdout: "addr": { 2023-09-12T14:55:23.115 INFO:teuthology.orchestra.run.smithi187.stdout: "type": "v1", 2023-09-12T14:55:23.116 INFO:teuthology.orchestra.run.smithi187.stdout: "addr": "192.168.0.2:0", 2023-09-12T14:55:23.116 INFO:teuthology.orchestra.run.smithi187.stdout: "nonce": 3008562153 2023-09-12T14:55:23.116 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-09-12T14:55:23.116 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-09-12T14:55:23.116 INFO:teuthology.orchestra.run.smithi187.stdout: "addr": { 2023-09-12T14:55:23.116 INFO:teuthology.orchestra.run.smithi187.stdout: "type": "v1", 2023-09-12T14:55:23.117 INFO:teuthology.orchestra.run.smithi187.stdout: "addr": "192.168.0.2:0", 2023-09-12T14:55:23.117 INFO:teuthology.orchestra.run.smithi187.stdout: "nonce": 3008562153 2023-09-12T14:55:23.117 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-09-12T14:55:23.118 INFO:teuthology.orchestra.run.smithi187.stdout: "inst_str": "client.4499 192.168.0.2:0/3008562153", 2023-09-12T14:55:23.118 INFO:teuthology.orchestra.run.smithi187.stdout: "addr_str": "192.168.0.2:0/3008562153", 2023-09-12T14:55:23.118 INFO:teuthology.orchestra.run.smithi187.stdout: "inode_count": 1, 2023-09-12T14:55:23.118 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_epoch": 6, 2023-09-12T14:55:23.119 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_epoch": 20, 2023-09-12T14:55:23.119 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_epoch_barrier": 0, 2023-09-12T14:55:23.119 INFO:teuthology.orchestra.run.smithi187.stdout: "blocklisted": false, 2023-09-12T14:55:23.119 INFO:teuthology.orchestra.run.smithi187.stdout: "fs_name": "cephfs" 2023-09-12T14:55:23.119 INFO:teuthology.orchestra.run.smithi187.stdout:} 2023-09-12T14:55:23.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:23.124 DEBUG:teuthology.orchestra.run.smithi131:> stat --file-system '--printf=%T 2023-09-12T14:55:23.125 DEBUG:teuthology.orchestra.run.smithi131:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-12T14:55:23.132 INFO:teuthology.orchestra.run.smithi131.stdout:fuseblk 2023-09-12T14:55:23.133 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-12T14:55:23.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:23.133 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-12T14:55:23.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:23.191 DEBUG:teuthology.orchestra.run.smithi131:> stat --file-system '--printf=%T 2023-09-12T14:55:23.191 DEBUG:teuthology.orchestra.run.smithi131:> ' -- /home/ubuntu/cephtest/mnt.2 2023-09-12T14:55:23.239 INFO:teuthology.orchestra.run.smithi131.stdout:fuseblk 2023-09-12T14:55:23.240 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-09-12T14:55:23.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:23.240 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-09-12T14:55:23.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:23.300 DEBUG:teuthology.orchestra.run.smithi187:> stat --file-system '--printf=%T 2023-09-12T14:55:23.300 DEBUG:teuthology.orchestra.run.smithi187:> ' -- /home/ubuntu/cephtest/mnt.1 2023-09-12T14:55:23.307 INFO:teuthology.orchestra.run.smithi187.stdout:fuseblk 2023-09-12T14:55:23.308 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-09-12T14:55:23.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:23.308 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-09-12T14:55:23.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:23.367 DEBUG:teuthology.orchestra.run.smithi187:> stat --file-system '--printf=%T 2023-09-12T14:55:23.367 DEBUG:teuthology.orchestra.run.smithi187:> ' -- /home/ubuntu/cephtest/mnt.3 2023-09-12T14:55:23.416 INFO:teuthology.orchestra.run.smithi187.stdout:fuseblk 2023-09-12T14:55:23.416 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-09-12T14:55:23.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:23.417 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-09-12T14:55:23.471 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-09-12T14:55:23.488 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:55:23.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9ee278640 1 -- 172.21.15.131:0/502584088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9e8156e80 msgr2=0x7fd9e806eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:23.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9ee278640 1 --2- 172.21.15.131:0/502584088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9e8156e80 0x7fd9e806eca0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80ab950 tx=0x7fd9d801c7e0 comp rx=0 tx=0).stop 2023-09-12T14:55:23.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9ee278640 1 -- 172.21.15.131:0/502584088 shutdown_connections 2023-09-12T14:55:23.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9ee278640 1 --2- 172.21.15.131:0/502584088 >> v2:172.21.15.187:3300/0 conn(0x7fd9e806baa0 0x7fd9e8157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:23.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9ee278640 1 --2- 172.21.15.131:0/502584088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9e8156e80 0x7fd9e806eca0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:23.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9ee278640 1 -- 172.21.15.131:0/502584088 >> 172.21.15.131:0/502584088 conn(0x7fd9e8069840 msgr2=0x7fd9e80cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:23.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9ee278640 1 -- 172.21.15.131:0/502584088 shutdown_connections 2023-09-12T14:55:23.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9ee278640 1 -- 172.21.15.131:0/502584088 wait complete. 2023-09-12T14:55:23.626 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9ee278640 1 Processor -- start 2023-09-12T14:55:23.626 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9ee278640 1 -- start start 2023-09-12T14:55:23.626 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9ee278640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9e806baa0 0x7fd9e81432b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:23.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9ee278640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9e806f1e0 0x7fd9e8160840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:23.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9ee278640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9e8156e80 0x7fd9e80aa270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:23.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9ee278640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e81437f0 con 0x7fd9e806f1e0 2023-09-12T14:55:23.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9ee278640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e8160d80 con 0x7fd9e806baa0 2023-09-12T14:55:23.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9ee278640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e80a8460 con 0x7fd9e8156e80 2023-09-12T14:55:23.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9e77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9e806f1e0 0x7fd9e8160840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:23.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9e77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9e806f1e0 0x7fd9e8160840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55842/0 (socket says 172.21.15.131:55842) 2023-09-12T14:55:23.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9e77fe640 1 -- 172.21.15.131:0/2645971062 learned_addr learned my addr 172.21.15.131:0/2645971062 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:23.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9e7fff640 1 --2- 172.21.15.131:0/2645971062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9e806baa0 0x7fd9e81432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:23.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9ec815640 1 --2- 172.21.15.131:0/2645971062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9e8156e80 0x7fd9e80aa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:23.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9e77fe640 1 -- 172.21.15.131:0/2645971062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9e8156e80 msgr2=0x7fd9e80aa270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:23.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9e77fe640 1 --2- 172.21.15.131:0/2645971062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9e8156e80 0x7fd9e80aa270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:23.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9e77fe640 1 -- 172.21.15.131:0/2645971062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9e806baa0 msgr2=0x7fd9e81432b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:23.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9e77fe640 1 --2- 172.21.15.131:0/2645971062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9e806baa0 0x7fd9e81432b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:23.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9e77fe640 1 -- 172.21.15.131:0/2645971062 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d8009d30 con 0x7fd9e806f1e0 2023-09-12T14:55:23.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9ec815640 1 --2- 172.21.15.131:0/2645971062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9e8156e80 0x7fd9e80aa270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:55:23.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9e77fe640 1 --2- 172.21.15.131:0/2645971062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9e806f1e0 0x7fd9e8160840 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000cbc0 tx=0x7fd9d000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:23.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9e4ff9640 1 -- 172.21.15.131:0/2645971062 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0003b80 con 0x7fd9e806f1e0 2023-09-12T14:55:23.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9e4ff9640 1 -- 172.21.15.131:0/2645971062 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd9d0003dd0 con 0x7fd9e806f1e0 2023-09-12T14:55:23.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9ee278640 1 -- 172.21.15.131:0/2645971062 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e80a8740 con 0x7fd9e806f1e0 2023-09-12T14:55:23.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9e4ff9640 1 -- 172.21.15.131:0/2645971062 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d000aa50 con 0x7fd9e806f1e0 2023-09-12T14:55:23.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.621+0000 7fd9ee278640 1 -- 172.21.15.131:0/2645971062 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e80a8d00 con 0x7fd9e806f1e0 2023-09-12T14:55:23.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.625+0000 7fd9ee278640 1 -- 172.21.15.131:0/2645971062 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e8073580 con 0x7fd9e806f1e0 2023-09-12T14:55:23.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.625+0000 7fd9e4ff9640 1 -- 172.21.15.131:0/2645971062 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fd9d000abb0 con 0x7fd9e806f1e0 2023-09-12T14:55:23.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.625+0000 7fd9e4ff9640 1 --2- 172.21.15.131:0/2645971062 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd9c8065b20 0x7fd9c8067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:23.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.625+0000 7fd9e4ff9640 1 -- 172.21.15.131:0/2645971062 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9d0083930 con 0x7fd9e806f1e0 2023-09-12T14:55:23.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.625+0000 7fd9e7fff640 1 --2- 172.21.15.131:0/2645971062 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd9c8065b20 0x7fd9c8067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:23.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.625+0000 7fd9e7fff640 1 --2- 172.21.15.131:0/2645971062 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd9c8065b20 0x7fd9c8067fe0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80af000 tx=0x7fd9d801b020 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:23.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.629+0000 7fd9e4ff9640 1 -- 172.21.15.131:0/2645971062 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fd9d0051280 con 0x7fd9e806f1e0 2023-09-12T14:55:23.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.781+0000 7fd9ee278640 1 -- 172.21.15.131:0/2645971062 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9e80cb9f0 con 0x7fd9e806f1e0 2023-09-12T14:55:23.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.785+0000 7fd9e4ff9640 1 -- 172.21.15.131:0/2645971062 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fd9d000ae60 con 0x7fd9e806f1e0 2023-09-12T14:55:23.787 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:23.788 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:54:59.694778+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:54:55.689983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T14:54:57.029447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:55:23.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.785+0000 7fd9ee278640 1 -- 172.21.15.131:0/2645971062 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd9c8065b20 msgr2=0x7fd9c8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:23.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.785+0000 7fd9ee278640 1 --2- 172.21.15.131:0/2645971062 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd9c8065b20 0x7fd9c8067fe0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80af000 tx=0x7fd9d801b020 comp rx=0 tx=0).stop 2023-09-12T14:55:23.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.785+0000 7fd9ee278640 1 -- 172.21.15.131:0/2645971062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9e806f1e0 msgr2=0x7fd9e8160840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:23.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.785+0000 7fd9ee278640 1 --2- 172.21.15.131:0/2645971062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9e806f1e0 0x7fd9e8160840 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000cbc0 tx=0x7fd9d000ccb0 comp rx=0 tx=0).stop 2023-09-12T14:55:23.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.785+0000 7fd9ee278640 1 -- 172.21.15.131:0/2645971062 shutdown_connections 2023-09-12T14:55:23.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.785+0000 7fd9ee278640 1 --2- 172.21.15.131:0/2645971062 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd9c8065b20 0x7fd9c8067fe0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:23.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.785+0000 7fd9ee278640 1 --2- 172.21.15.131:0/2645971062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9e8156e80 0x7fd9e80aa270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:23.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.785+0000 7fd9ee278640 1 --2- 172.21.15.131:0/2645971062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9e806f1e0 0x7fd9e8160840 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:23.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.785+0000 7fd9ee278640 1 --2- 172.21.15.131:0/2645971062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9e806baa0 0x7fd9e81432b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:23.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.785+0000 7fd9ee278640 1 -- 172.21.15.131:0/2645971062 >> 172.21.15.131:0/2645971062 conn(0x7fd9e8069840 msgr2=0x7fd9e81547b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:23.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.785+0000 7fd9ee278640 1 -- 172.21.15.131:0/2645971062 shutdown_connections 2023-09-12T14:55:23.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.785+0000 7fd9ee278640 1 -- 172.21.15.131:0/2645971062 wait complete. 2023-09-12T14:55:23.805 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T14:54:45.654004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T14:54:55.689983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T14:54:57.029447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-12T14:55:23.806 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T14:55:23.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.921+0000 7f827b49b640 1 -- 172.21.15.131:0/577317173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8260004d10 msgr2=0x7f8260005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:23.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.921+0000 7f827b49b640 1 --2- 172.21.15.131:0/577317173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8260004d10 0x7f8260005190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f826c00c5c0 tx=0x7f826c01fac0 comp rx=0 tx=0).stop 2023-09-12T14:55:23.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.921+0000 7f8279a38640 1 -- 172.21.15.131:0/577317173 reap_dead start 2023-09-12T14:55:23.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.921+0000 7f827b49b640 1 -- 172.21.15.131:0/577317173 shutdown_connections 2023-09-12T14:55:23.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.921+0000 7f827b49b640 1 -- 172.21.15.131:0/577317173 >> 172.21.15.131:0/577317173 conn(0x7f82740cebe0 msgr2=0x7f82740ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:23.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.921+0000 7f827b49b640 1 -- 172.21.15.131:0/577317173 shutdown_connections 2023-09-12T14:55:23.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.921+0000 7f827b49b640 1 -- 172.21.15.131:0/577317173 wait complete. 2023-09-12T14:55:23.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.921+0000 7f827b49b640 1 Processor -- start 2023-09-12T14:55:23.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.921+0000 7f827b49b640 1 -- start start 2023-09-12T14:55:23.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.921+0000 7f827b49b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8260004d10 0x7f8274155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:23.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.921+0000 7f827b49b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82741efb60 0x7f82741f2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:23.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.921+0000 7f827b49b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82741effe0 0x7f8274156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:23.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.921+0000 7f827b49b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82741557a0 con 0x7f8260004d10 2023-09-12T14:55:23.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.921+0000 7f827b49b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82741f3110 con 0x7f82741efb60 2023-09-12T14:55:23.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.921+0000 7f827b49b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82741f33f0 con 0x7f82741effe0 2023-09-12T14:55:23.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.921+0000 7f8279237640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8260004d10 0x7f8274155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:23.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.921+0000 7f8279237640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8260004d10 0x7f8274155260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55856/0 (socket says 172.21.15.131:55856) 2023-09-12T14:55:23.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.921+0000 7f8279237640 1 -- 172.21.15.131:0/4681715 learned_addr learned my addr 172.21.15.131:0/4681715 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:23.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.925+0000 7f8279a38640 1 --2- 172.21.15.131:0/4681715 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82741effe0 0x7f8274156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:23.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.925+0000 7f8278a36640 1 --2- 172.21.15.131:0/4681715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82741efb60 0x7f82741f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:23.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.925+0000 7f8279237640 1 -- 172.21.15.131:0/4681715 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82741effe0 msgr2=0x7f8274156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:23.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.925+0000 7f8279237640 1 --2- 172.21.15.131:0/4681715 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82741effe0 0x7f8274156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:23.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.925+0000 7f8279237640 1 -- 172.21.15.131:0/4681715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82741efb60 msgr2=0x7f82741f2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:23.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.925+0000 7f8279237640 1 --2- 172.21.15.131:0/4681715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82741efb60 0x7f82741f2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:23.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.925+0000 7f8279237640 1 -- 172.21.15.131:0/4681715 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82741ef080 con 0x7f8260004d10 2023-09-12T14:55:23.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.925+0000 7f8279237640 1 --2- 172.21.15.131:0/4681715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8260004d10 0x7f8274155260 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f827000c500 tx=0x7f827000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:23.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.925+0000 7f8269ffb640 1 -- 172.21.15.131:0/4681715 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82700052d0 con 0x7f8260004d10 2023-09-12T14:55:23.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.925+0000 7f8269ffb640 1 -- 172.21.15.131:0/4681715 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8270005430 con 0x7f8260004d10 2023-09-12T14:55:23.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.925+0000 7f827b49b640 1 -- 172.21.15.131:0/4681715 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82741ef360 con 0x7f8260004d10 2023-09-12T14:55:23.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.925+0000 7f8269ffb640 1 -- 172.21.15.131:0/4681715 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82700038b0 con 0x7f8260004d10 2023-09-12T14:55:23.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.925+0000 7f827b49b640 1 -- 172.21.15.131:0/4681715 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f827415b200 con 0x7f8260004d10 2023-09-12T14:55:23.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.925+0000 7f827b49b640 1 -- 172.21.15.131:0/4681715 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8274073580 con 0x7f8260004d10 2023-09-12T14:55:23.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.925+0000 7f8269ffb640 1 -- 172.21.15.131:0/4681715 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f8270003a10 con 0x7f8260004d10 2023-09-12T14:55:23.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.925+0000 7f8269ffb640 1 --2- 172.21.15.131:0/4681715 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8258065b20 0x7f8258067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:23.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.925+0000 7f8278a36640 1 --2- 172.21.15.131:0/4681715 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8258065b20 0x7f8258067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:23.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.925+0000 7f8269ffb640 1 -- 172.21.15.131:0/4681715 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8270083060 con 0x7f8260004d10 2023-09-12T14:55:23.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.925+0000 7f8278a36640 1 --2- 172.21.15.131:0/4681715 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8258065b20 0x7f8258067fe0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f82740cb350 tx=0x7f82640091e0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:23.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:23.929+0000 7f8269ffb640 1 -- 172.21.15.131:0/4681715 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f8270050a40 con 0x7f8260004d10 2023-09-12T14:55:24.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.077+0000 7f827b49b640 1 -- 172.21.15.131:0/4681715 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f82740b4b60 con 0x7f8260004d10 2023-09-12T14:55:24.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.077+0000 7f8269ffb640 1 -- 172.21.15.131:0/4681715 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8270054060 con 0x7f8260004d10 2023-09-12T14:55:24.079 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-09-12T14:55:24.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.077+0000 7f827b49b640 1 -- 172.21.15.131:0/4681715 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8258065b20 msgr2=0x7f8258067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:24.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.077+0000 7f827b49b640 1 --2- 172.21.15.131:0/4681715 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8258065b20 0x7f8258067fe0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f82740cb350 tx=0x7f82640091e0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.077+0000 7f827b49b640 1 -- 172.21.15.131:0/4681715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8260004d10 msgr2=0x7f8274155260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:24.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.077+0000 7f827b49b640 1 --2- 172.21.15.131:0/4681715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8260004d10 0x7f8274155260 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f827000c500 tx=0x7f827000c9c0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.077+0000 7f827b49b640 1 -- 172.21.15.131:0/4681715 shutdown_connections 2023-09-12T14:55:24.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.077+0000 7f827b49b640 1 --2- 172.21.15.131:0/4681715 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8258065b20 0x7f8258067fe0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.077+0000 7f827b49b640 1 --2- 172.21.15.131:0/4681715 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82741effe0 0x7f8274156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.077+0000 7f827b49b640 1 --2- 172.21.15.131:0/4681715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82741efb60 0x7f82741f2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.077+0000 7f827b49b640 1 --2- 172.21.15.131:0/4681715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8260004d10 0x7f8274155260 secure :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f827000c500 tx=0x7f827000c9c0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.077+0000 7f827b49b640 1 -- 172.21.15.131:0/4681715 >> 172.21.15.131:0/4681715 conn(0x7f82740cebe0 msgr2=0x7f82740ca990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:24.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.077+0000 7f827b49b640 1 -- 172.21.15.131:0/4681715 shutdown_connections 2023-09-12T14:55:24.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.077+0000 7f827b49b640 1 -- 172.21.15.131:0/4681715 wait complete. 2023-09-12T14:55:24.096 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T14:55:24.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.217+0000 7fa51461a640 1 -- 172.21.15.131:0/1530606381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa50c0c9a20 msgr2=0x7fa50c15ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:24.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.217+0000 7fa51461a640 1 --2- 172.21.15.131:0/1530606381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa50c0c9a20 0x7fa50c15ca30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa504009c80 tx=0x7fa50401ccf0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.217+0000 7fa51461a640 1 -- 172.21.15.131:0/1530606381 shutdown_connections 2023-09-12T14:55:24.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.217+0000 7fa51461a640 1 --2- 172.21.15.131:0/1530606381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa50c0c9a20 0x7fa50c15ca30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.217+0000 7fa51461a640 1 --2- 172.21.15.131:0/1530606381 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa50c158d70 0x7fa50c06b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.217+0000 7fa51461a640 1 -- 172.21.15.131:0/1530606381 >> 172.21.15.131:0/1530606381 conn(0x7fa50c069840 msgr2=0x7fa50c0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:24.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.217+0000 7fa51461a640 1 -- 172.21.15.131:0/1530606381 shutdown_connections 2023-09-12T14:55:24.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.217+0000 7fa51461a640 1 -- 172.21.15.131:0/1530606381 wait complete. 2023-09-12T14:55:24.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.217+0000 7fa51461a640 1 Processor -- start 2023-09-12T14:55:24.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.217+0000 7fa51461a640 1 -- start start 2023-09-12T14:55:24.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa51461a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa50c0abae0 0x7fa50c1458b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:24.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa51461a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa50c0c9a20 0x7fa50c1460f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:24.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa51461a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa50c158d70 0x7fa50c14f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:24.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa51461a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c1473e0 con 0x7fa50c0abae0 2023-09-12T14:55:24.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa51461a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c146b40 con 0x7fa50c158d70 2023-09-12T14:55:24.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa51461a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c146e20 con 0x7fa50c0c9a20 2023-09-12T14:55:24.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa5123b6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa50c0abae0 0x7fa50c1458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:24.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa5123b6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa50c0abae0 0x7fa50c1458b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55864/0 (socket says 172.21.15.131:55864) 2023-09-12T14:55:24.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa5123b6640 1 -- 172.21.15.131:0/195000703 learned_addr learned my addr 172.21.15.131:0/195000703 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:24.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa512bb7640 1 --2- 172.21.15.131:0/195000703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa50c158d70 0x7fa50c14f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:24.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa511bb5640 1 --2- 172.21.15.131:0/195000703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa50c0c9a20 0x7fa50c1460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:24.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa5123b6640 1 -- 172.21.15.131:0/195000703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa50c0c9a20 msgr2=0x7fa50c1460f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:24.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa5123b6640 1 --2- 172.21.15.131:0/195000703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa50c0c9a20 0x7fa50c1460f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa5123b6640 1 -- 172.21.15.131:0/195000703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa50c158d70 msgr2=0x7fa50c14f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:24.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa5123b6640 1 --2- 172.21.15.131:0/195000703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa50c158d70 0x7fa50c14f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa5123b6640 1 -- 172.21.15.131:0/195000703 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5040098e0 con 0x7fa50c0abae0 2023-09-12T14:55:24.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa512bb7640 1 --2- 172.21.15.131:0/195000703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa50c158d70 0x7fa50c14f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:55:24.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa5123b6640 1 --2- 172.21.15.131:0/195000703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa50c0abae0 0x7fa50c1458b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa50800e7a0 tx=0x7fa50800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:24.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa502ffd640 1 -- 172.21.15.131:0/195000703 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa508009590 con 0x7fa50c0abae0 2023-09-12T14:55:24.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa502ffd640 1 -- 172.21.15.131:0/195000703 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa50800cdb0 con 0x7fa50c0abae0 2023-09-12T14:55:24.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa51461a640 1 -- 172.21.15.131:0/195000703 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa50c147100 con 0x7fa50c0abae0 2023-09-12T14:55:24.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa502ffd640 1 -- 172.21.15.131:0/195000703 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa508010690 con 0x7fa50c0abae0 2023-09-12T14:55:24.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa51461a640 1 -- 172.21.15.131:0/195000703 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa50c145df0 con 0x7fa50c0abae0 2023-09-12T14:55:24.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa51461a640 1 -- 172.21.15.131:0/195000703 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa50c073580 con 0x7fa50c0abae0 2023-09-12T14:55:24.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa502ffd640 1 -- 172.21.15.131:0/195000703 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fa5080107f0 con 0x7fa50c0abae0 2023-09-12T14:55:24.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa502ffd640 1 --2- 172.21.15.131:0/195000703 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa4f0065b20 0x7fa4f0067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:24.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa511bb5640 1 --2- 172.21.15.131:0/195000703 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa4f0065b20 0x7fa4f0067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:24.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa502ffd640 1 -- 172.21.15.131:0/195000703 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa508014070 con 0x7fa50c0abae0 2023-09-12T14:55:24.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.221+0000 7fa511bb5640 1 --2- 172.21.15.131:0/195000703 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa4f0065b20 0x7fa4f0067fe0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa50c157c90 tx=0x7fa4fc009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:24.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.225+0000 7fa502ffd640 1 -- 172.21.15.131:0/195000703 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fa50804fdb0 con 0x7fa50c0abae0 2023-09-12T14:55:24.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.364+0000 7fa51461a640 1 -- 172.21.15.131:0/195000703 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa50c0cd4d0 con 0x7fa50c0abae0 2023-09-12T14:55:24.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.368+0000 7fa502ffd640 1 -- 172.21.15.131:0/195000703 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa5080516e0 con 0x7fa50c0abae0 2023-09-12T14:55:24.370 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-09-12T14:55:24.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.368+0000 7fa51461a640 1 -- 172.21.15.131:0/195000703 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa4f0065b20 msgr2=0x7fa4f0067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:24.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.368+0000 7fa51461a640 1 --2- 172.21.15.131:0/195000703 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa4f0065b20 0x7fa4f0067fe0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa50c157c90 tx=0x7fa4fc009040 comp rx=0 tx=0).stop 2023-09-12T14:55:24.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.368+0000 7fa51461a640 1 -- 172.21.15.131:0/195000703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa50c0abae0 msgr2=0x7fa50c1458b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:24.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.368+0000 7fa51461a640 1 --2- 172.21.15.131:0/195000703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa50c0abae0 0x7fa50c1458b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa50800e7a0 tx=0x7fa50800ec60 comp rx=0 tx=0).stop 2023-09-12T14:55:24.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.368+0000 7fa51461a640 1 -- 172.21.15.131:0/195000703 shutdown_connections 2023-09-12T14:55:24.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.368+0000 7fa51461a640 1 --2- 172.21.15.131:0/195000703 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa4f0065b20 0x7fa4f0067fe0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.368+0000 7fa51461a640 1 --2- 172.21.15.131:0/195000703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa50c0c9a20 0x7fa50c1460f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.368+0000 7fa51461a640 1 --2- 172.21.15.131:0/195000703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa50c158d70 0x7fa50c14f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.368+0000 7fa51461a640 1 --2- 172.21.15.131:0/195000703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa50c0abae0 0x7fa50c1458b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.368+0000 7fa51461a640 1 -- 172.21.15.131:0/195000703 >> 172.21.15.131:0/195000703 conn(0x7fa50c069840 msgr2=0x7fa50c0ae690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:24.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.368+0000 7fa51461a640 1 -- 172.21.15.131:0/195000703 shutdown_connections 2023-09-12T14:55:24.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.368+0000 7fa51461a640 1 -- 172.21.15.131:0/195000703 wait complete. 2023-09-12T14:55:24.388 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T14:55:24.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc27b64a640 1 -- 172.21.15.131:0/4251146085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc260004d10 msgr2=0x7fc260005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:24.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc27b64a640 1 --2- 172.21.15.131:0/4251146085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc260004d10 0x7fc260005190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc26c009e60 tx=0x7fc26c01d130 comp rx=0 tx=0).stop 2023-09-12T14:55:24.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc279be7640 1 -- 172.21.15.131:0/4251146085 reap_dead start 2023-09-12T14:55:24.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc27b64a640 1 -- 172.21.15.131:0/4251146085 shutdown_connections 2023-09-12T14:55:24.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc27b64a640 1 -- 172.21.15.131:0/4251146085 >> 172.21.15.131:0/4251146085 conn(0x7fc274069bb0 msgr2=0x7fc274069f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:24.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc27b64a640 1 -- 172.21.15.131:0/4251146085 shutdown_connections 2023-09-12T14:55:24.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc27b64a640 1 -- 172.21.15.131:0/4251146085 wait complete. 2023-09-12T14:55:24.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc27b64a640 1 Processor -- start 2023-09-12T14:55:24.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc27b64a640 1 -- start start 2023-09-12T14:55:24.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc27b64a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc27414a580 0x7fc2740b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:24.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc27b64a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc274158d70 0x7fc2741591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:24.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc27b64a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc27414f440 0x7fc27414f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:24.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc27b64a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2740b2d60 con 0x7fc27414a580 2023-09-12T14:55:24.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc27b64a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc274159730 con 0x7fc27414f440 2023-09-12T14:55:24.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc27b64a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2740b20e0 con 0x7fc274158d70 2023-09-12T14:55:24.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc2793e6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc27414a580 0x7fc2740b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:24.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc2793e6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc27414a580 0x7fc2740b1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55878/0 (socket says 172.21.15.131:55878) 2023-09-12T14:55:24.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc2793e6640 1 -- 172.21.15.131:0/4172390158 learned_addr learned my addr 172.21.15.131:0/4172390158 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:24.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc279be7640 1 --2- 172.21.15.131:0/4172390158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc27414f440 0x7fc27414f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:24.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc278be5640 1 --2- 172.21.15.131:0/4172390158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc274158d70 0x7fc2741591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:24.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc2793e6640 1 -- 172.21.15.131:0/4172390158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc274158d70 msgr2=0x7fc2741591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:24.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc2793e6640 1 --2- 172.21.15.131:0/4172390158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc274158d70 0x7fc2741591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc2793e6640 1 -- 172.21.15.131:0/4172390158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc27414f440 msgr2=0x7fc27414f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:24.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc2793e6640 1 --2- 172.21.15.131:0/4172390158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc27414f440 0x7fc27414f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc2793e6640 1 -- 172.21.15.131:0/4172390158 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2740b23c0 con 0x7fc27414a580 2023-09-12T14:55:24.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc278be5640 1 --2- 172.21.15.131:0/4172390158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc274158d70 0x7fc2741591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:55:24.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc279be7640 1 --2- 172.21.15.131:0/4172390158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc27414f440 0x7fc27414f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:55:24.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc2793e6640 1 --2- 172.21.15.131:0/4172390158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc27414a580 0x7fc2740b1890 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc270003680 tx=0x7fc27000ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:24.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc269ffb640 1 -- 172.21.15.131:0/4172390158 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27000c910 con 0x7fc27414a580 2023-09-12T14:55:24.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc269ffb640 1 -- 172.21.15.131:0/4172390158 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc27000cb60 con 0x7fc27414a580 2023-09-12T14:55:24.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc27b64a640 1 -- 172.21.15.131:0/4172390158 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2740b26a0 con 0x7fc27414a580 2023-09-12T14:55:24.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.508+0000 7fc269ffb640 1 -- 172.21.15.131:0/4172390158 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc270011460 con 0x7fc27414a580 2023-09-12T14:55:24.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.512+0000 7fc27b64a640 1 -- 172.21.15.131:0/4172390158 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2740b1dd0 con 0x7fc27414a580 2023-09-12T14:55:24.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.512+0000 7fc27b64a640 1 -- 172.21.15.131:0/4172390158 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc274073580 con 0x7fc27414a580 2023-09-12T14:55:24.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.512+0000 7fc269ffb640 1 -- 172.21.15.131:0/4172390158 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fc2700115c0 con 0x7fc27414a580 2023-09-12T14:55:24.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.512+0000 7fc269ffb640 1 --2- 172.21.15.131:0/4172390158 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc258065b20 0x7fc258067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:24.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.512+0000 7fc269ffb640 1 -- 172.21.15.131:0/4172390158 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc270083b50 con 0x7fc27414a580 2023-09-12T14:55:24.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.512+0000 7fc278be5640 1 --2- 172.21.15.131:0/4172390158 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc258065b20 0x7fc258067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:24.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.512+0000 7fc278be5640 1 --2- 172.21.15.131:0/4172390158 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc258065b20 0x7fc258067fe0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc2741599b0 tx=0x7fc264006c50 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:24.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.516+0000 7fc269ffb640 1 -- 172.21.15.131:0/4172390158 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fc270051510 con 0x7fc27414a580 2023-09-12T14:55:24.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.656+0000 7fc27b64a640 1 -- 172.21.15.131:0/4172390158 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc27406a8c0 con 0x7fc27414a580 2023-09-12T14:55:24.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.656+0000 7fc269ffb640 1 -- 172.21.15.131:0/4172390158 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc27001f0a0 con 0x7fc27414a580 2023-09-12T14:55:24.660 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-09-12T14:55:24.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.656+0000 7fc27b64a640 1 -- 172.21.15.131:0/4172390158 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc258065b20 msgr2=0x7fc258067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:24.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.656+0000 7fc27b64a640 1 --2- 172.21.15.131:0/4172390158 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc258065b20 0x7fc258067fe0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc2741599b0 tx=0x7fc264006c50 comp rx=0 tx=0).stop 2023-09-12T14:55:24.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.656+0000 7fc27b64a640 1 -- 172.21.15.131:0/4172390158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc27414a580 msgr2=0x7fc2740b1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:24.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.656+0000 7fc27b64a640 1 --2- 172.21.15.131:0/4172390158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc27414a580 0x7fc2740b1890 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc270003680 tx=0x7fc27000ea90 comp rx=0 tx=0).stop 2023-09-12T14:55:24.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.656+0000 7fc27b64a640 1 -- 172.21.15.131:0/4172390158 shutdown_connections 2023-09-12T14:55:24.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.656+0000 7fc27b64a640 1 --2- 172.21.15.131:0/4172390158 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc258065b20 0x7fc258067fe0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.656+0000 7fc27b64a640 1 --2- 172.21.15.131:0/4172390158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc274158d70 0x7fc2741591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.656+0000 7fc27b64a640 1 --2- 172.21.15.131:0/4172390158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc27414f440 0x7fc27414f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.656+0000 7fc27b64a640 1 --2- 172.21.15.131:0/4172390158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc27414a580 0x7fc2740b1890 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.656+0000 7fc27b64a640 1 -- 172.21.15.131:0/4172390158 >> 172.21.15.131:0/4172390158 conn(0x7fc274069bb0 msgr2=0x7fc274157230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:24.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.660+0000 7fc27b64a640 1 -- 172.21.15.131:0/4172390158 shutdown_connections 2023-09-12T14:55:24.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.660+0000 7fc27b64a640 1 -- 172.21.15.131:0/4172390158 wait complete. 2023-09-12T14:55:24.677 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:55:24.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.788+0000 7fda831fd640 1 -- 172.21.15.131:0/2125631538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda7c15e9f0 msgr2=0x7fda7c0b0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:24.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.788+0000 7fda831fd640 1 --2- 172.21.15.131:0/2125631538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda7c15e9f0 0x7fda7c0b0e60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fda74009bc0 tx=0x7fda7401cb10 comp rx=0 tx=0).stop 2023-09-12T14:55:24.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.788+0000 7fda831fd640 1 -- 172.21.15.131:0/2125631538 shutdown_connections 2023-09-12T14:55:24.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.788+0000 7fda831fd640 1 --2- 172.21.15.131:0/2125631538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda7c15e9f0 0x7fda7c0b0e60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.788+0000 7fda831fd640 1 -- 172.21.15.131:0/2125631538 >> 172.21.15.131:0/2125631538 conn(0x7fda7c09a200 msgr2=0x7fda7c0d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:24.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.788+0000 7fda831fd640 1 -- 172.21.15.131:0/2125631538 shutdown_connections 2023-09-12T14:55:24.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.788+0000 7fda831fd640 1 -- 172.21.15.131:0/2125631538 wait complete. 2023-09-12T14:55:24.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.788+0000 7fda831fd640 1 Processor -- start 2023-09-12T14:55:24.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.788+0000 7fda831fd640 1 -- start start 2023-09-12T14:55:24.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.788+0000 7fda831fd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda7c0cf2d0 0x7fda7c0c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:24.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.788+0000 7fda831fd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda7c0d1680 0x7fda7c0b62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:24.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.788+0000 7fda831fd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda7c15e9f0 0x7fda7c0c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:24.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.788+0000 7fda831fd640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c14a540 con 0x7fda7c0d1680 2023-09-12T14:55:24.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.788+0000 7fda831fd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c154e00 con 0x7fda7c15e9f0 2023-09-12T14:55:24.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.788+0000 7fda831fd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c1550e0 con 0x7fda7c0cf2d0 2023-09-12T14:55:24.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.788+0000 7fda7bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda7c0d1680 0x7fda7c0b62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:24.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.788+0000 7fda7bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda7c0d1680 0x7fda7c0b62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55894/0 (socket says 172.21.15.131:55894) 2023-09-12T14:55:24.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.788+0000 7fda7bfff640 1 -- 172.21.15.131:0/3446243292 learned_addr learned my addr 172.21.15.131:0/3446243292 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:24.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.788+0000 7fda7bfff640 1 -- 172.21.15.131:0/3446243292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda7c0cf2d0 msgr2=0x7fda7c0c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:24.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.788+0000 7fda7bfff640 1 --2- 172.21.15.131:0/3446243292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda7c0cf2d0 0x7fda7c0c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.788+0000 7fda7bfff640 1 -- 172.21.15.131:0/3446243292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda7c15e9f0 msgr2=0x7fda7c0c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:24.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.788+0000 7fda7bfff640 1 --2- 172.21.15.131:0/3446243292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda7c15e9f0 0x7fda7c0c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.788+0000 7fda7bfff640 1 -- 172.21.15.131:0/3446243292 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda74009820 con 0x7fda7c0d1680 2023-09-12T14:55:24.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.788+0000 7fda7bfff640 1 --2- 172.21.15.131:0/3446243292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda7c0d1680 0x7fda7c0b62f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fda6800c480 tx=0x7fda6800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:24.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.788+0000 7fda797fa640 1 -- 172.21.15.131:0/3446243292 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda68005480 con 0x7fda7c0d1680 2023-09-12T14:55:24.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.788+0000 7fda797fa640 1 -- 172.21.15.131:0/3446243292 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fda68003600 con 0x7fda7c0d1680 2023-09-12T14:55:24.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.792+0000 7fda831fd640 1 -- 172.21.15.131:0/3446243292 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda7c1553c0 con 0x7fda7c0d1680 2023-09-12T14:55:24.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.792+0000 7fda797fa640 1 -- 172.21.15.131:0/3446243292 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda68003a10 con 0x7fda7c0d1680 2023-09-12T14:55:24.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.792+0000 7fda831fd640 1 -- 172.21.15.131:0/3446243292 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7c0c71b0 con 0x7fda7c0d1680 2023-09-12T14:55:24.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.792+0000 7fda831fd640 1 -- 172.21.15.131:0/3446243292 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda7c073580 con 0x7fda7c0d1680 2023-09-12T14:55:24.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.792+0000 7fda797fa640 1 -- 172.21.15.131:0/3446243292 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fda68003b70 con 0x7fda7c0d1680 2023-09-12T14:55:24.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.792+0000 7fda797fa640 1 --2- 172.21.15.131:0/3446243292 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fda60065b50 0x7fda60068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:24.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.792+0000 7fda797fa640 1 -- 172.21.15.131:0/3446243292 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda68082f90 con 0x7fda7c0d1680 2023-09-12T14:55:24.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.792+0000 7fda80f99640 1 --2- 172.21.15.131:0/3446243292 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fda60065b50 0x7fda60068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:24.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.792+0000 7fda80f99640 1 --2- 172.21.15.131:0/3446243292 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fda60065b50 0x7fda60068010 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fda70003cf0 tx=0x7fda700079c0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:24.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.796+0000 7fda797fa640 1 -- 172.21.15.131:0/3446243292 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fda680b3db0 con 0x7fda7c0d1680 2023-09-12T14:55:24.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.948+0000 7fda831fd640 1 -- 172.21.15.131:0/3446243292 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda7c0b0000 con 0x7fda7c0d1680 2023-09-12T14:55:24.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.952+0000 7fda797fa640 1 -- 172.21.15.131:0/3446243292 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fda68050970 con 0x7fda7c0d1680 2023-09-12T14:55:24.955 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:24.956 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:54:59.694778+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:54:55.689983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T14:54:57.029447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:55:24.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.952+0000 7fda831fd640 1 -- 172.21.15.131:0/3446243292 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fda60065b50 msgr2=0x7fda60068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:24.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.952+0000 7fda831fd640 1 --2- 172.21.15.131:0/3446243292 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fda60065b50 0x7fda60068010 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fda70003cf0 tx=0x7fda700079c0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.952+0000 7fda831fd640 1 -- 172.21.15.131:0/3446243292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda7c0d1680 msgr2=0x7fda7c0b62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:24.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.952+0000 7fda831fd640 1 --2- 172.21.15.131:0/3446243292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda7c0d1680 0x7fda7c0b62f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fda6800c480 tx=0x7fda6800c940 comp rx=0 tx=0).stop 2023-09-12T14:55:24.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.952+0000 7fda831fd640 1 -- 172.21.15.131:0/3446243292 shutdown_connections 2023-09-12T14:55:24.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.952+0000 7fda831fd640 1 --2- 172.21.15.131:0/3446243292 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fda60065b50 0x7fda60068010 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.952+0000 7fda831fd640 1 --2- 172.21.15.131:0/3446243292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda7c15e9f0 0x7fda7c0c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.952+0000 7fda831fd640 1 --2- 172.21.15.131:0/3446243292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda7c0d1680 0x7fda7c0b62f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.956+0000 7fda831fd640 1 --2- 172.21.15.131:0/3446243292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda7c0cf2d0 0x7fda7c0c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:24.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.956+0000 7fda831fd640 1 -- 172.21.15.131:0/3446243292 >> 172.21.15.131:0/3446243292 conn(0x7fda7c09a200 msgr2=0x7fda7c0b2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:24.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.956+0000 7fda831fd640 1 -- 172.21.15.131:0/3446243292 shutdown_connections 2023-09-12T14:55:24.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:24.956+0000 7fda831fd640 1 -- 172.21.15.131:0/3446243292 wait complete. 2023-09-12T14:55:24.975 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T14:54:45.654004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T14:54:55.689983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T14:54:57.029447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-12T14:55:24.975 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T14:55:25.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.096+0000 7fb6ff510640 1 -- 172.21.15.131:0/3529425688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6e4004d10 msgr2=0x7fb6e4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:25.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.096+0000 7fb6ff510640 1 --2- 172.21.15.131:0/3529425688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6e4004d10 0x7fb6e4005190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4009ee0 tx=0x7fb6f401d5d0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.096+0000 7fb6fdaad640 1 -- 172.21.15.131:0/3529425688 reap_dead start 2023-09-12T14:55:25.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.096+0000 7fb6ff510640 1 -- 172.21.15.131:0/3529425688 shutdown_connections 2023-09-12T14:55:25.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.096+0000 7fb6ff510640 1 -- 172.21.15.131:0/3529425688 >> 172.21.15.131:0/3529425688 conn(0x7fb6f8069840 msgr2=0x7fb6f80cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:25.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.096+0000 7fb6ff510640 1 -- 172.21.15.131:0/3529425688 shutdown_connections 2023-09-12T14:55:25.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.096+0000 7fb6ff510640 1 -- 172.21.15.131:0/3529425688 wait complete. 2023-09-12T14:55:25.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.096+0000 7fb6ff510640 1 Processor -- start 2023-09-12T14:55:25.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.096+0000 7fb6ff510640 1 -- start start 2023-09-12T14:55:25.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.096+0000 7fb6ff510640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6e4004d10 0x7fb6f8150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:25.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.096+0000 7fb6ff510640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6f814d700 0x7fb6f8140440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:25.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.096+0000 7fb6ff510640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6f80ae690 0x7fb6f80aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:25.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.096+0000 7fb6ff510640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f81f01c0 con 0x7fb6f80ae690 2023-09-12T14:55:25.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.096+0000 7fb6ff510640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f80af060 con 0x7fb6f814d700 2023-09-12T14:55:25.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.096+0000 7fb6ff510640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f81ef540 con 0x7fb6e4004d10 2023-09-12T14:55:25.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.100+0000 7fb6fdaad640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6f80ae690 0x7fb6f80aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:25.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.100+0000 7fb6fdaad640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6f80ae690 0x7fb6f80aeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55916/0 (socket says 172.21.15.131:55916) 2023-09-12T14:55:25.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.100+0000 7fb6fdaad640 1 -- 172.21.15.131:0/214370938 learned_addr learned my addr 172.21.15.131:0/214370938 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:25.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.100+0000 7fb6fd2ac640 1 --2- 172.21.15.131:0/214370938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6e4004d10 0x7fb6f8150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:25.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.100+0000 7fb6fcaab640 1 --2- 172.21.15.131:0/214370938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6f814d700 0x7fb6f8140440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:25.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.100+0000 7fb6fdaad640 1 -- 172.21.15.131:0/214370938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6e4004d10 msgr2=0x7fb6f8150940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:25.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.100+0000 7fb6fdaad640 1 --2- 172.21.15.131:0/214370938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6e4004d10 0x7fb6f8150940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.100+0000 7fb6fdaad640 1 -- 172.21.15.131:0/214370938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6f814d700 msgr2=0x7fb6f8140440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:25.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.100+0000 7fb6fdaad640 1 --2- 172.21.15.131:0/214370938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6f814d700 0x7fb6f8140440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.100+0000 7fb6fdaad640 1 -- 172.21.15.131:0/214370938 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f81ef820 con 0x7fb6f80ae690 2023-09-12T14:55:25.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.100+0000 7fb6fd2ac640 1 --2- 172.21.15.131:0/214370938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6e4004d10 0x7fb6f8150940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:25.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.100+0000 7fb6fdaad640 1 --2- 172.21.15.131:0/214370938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6f80ae690 0x7fb6f80aeb20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4009ee0 tx=0x7fb6f4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:25.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.100+0000 7fb6edffb640 1 -- 172.21.15.131:0/214370938 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f40b6070 con 0x7fb6f80ae690 2023-09-12T14:55:25.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.100+0000 7fb6edffb640 1 -- 172.21.15.131:0/214370938 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb6f40a9510 con 0x7fb6f80ae690 2023-09-12T14:55:25.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.100+0000 7fb6ff510640 1 -- 172.21.15.131:0/214370938 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f81efb00 con 0x7fb6f80ae690 2023-09-12T14:55:25.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.100+0000 7fb6edffb640 1 -- 172.21.15.131:0/214370938 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f40a9c00 con 0x7fb6f80ae690 2023-09-12T14:55:25.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.100+0000 7fb6ff510640 1 -- 172.21.15.131:0/214370938 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f81ef240 con 0x7fb6f80ae690 2023-09-12T14:55:25.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.100+0000 7fb6ff510640 1 -- 172.21.15.131:0/214370938 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f8073580 con 0x7fb6f80ae690 2023-09-12T14:55:25.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.100+0000 7fb6edffb640 1 -- 172.21.15.131:0/214370938 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fb6f4005040 con 0x7fb6f80ae690 2023-09-12T14:55:25.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.100+0000 7fb6edffb640 1 --2- 172.21.15.131:0/214370938 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb6dc065b20 0x7fb6dc067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:25.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.100+0000 7fb6fd2ac640 1 --2- 172.21.15.131:0/214370938 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb6dc065b20 0x7fb6dc067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:25.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.100+0000 7fb6edffb640 1 -- 172.21.15.131:0/214370938 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6f411db20 con 0x7fb6f80ae690 2023-09-12T14:55:25.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.100+0000 7fb6fd2ac640 1 --2- 172.21.15.131:0/214370938 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb6dc065b20 0x7fb6dc067fe0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0001060 tx=0x7fb6f00067b0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:25.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.104+0000 7fb6edffb640 1 -- 172.21.15.131:0/214370938 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fb6f40eb500 con 0x7fb6f80ae690 2023-09-12T14:55:25.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.248+0000 7fb6ff510640 1 -- 172.21.15.131:0/214370938 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb6f80cd4d0 con 0x7fb6f80ae690 2023-09-12T14:55:25.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.248+0000 7fb6edffb640 1 -- 172.21.15.131:0/214370938 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb6f40c00a0 con 0x7fb6f80ae690 2023-09-12T14:55:25.253 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-09-12T14:55:25.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.252+0000 7fb6ff510640 1 -- 172.21.15.131:0/214370938 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb6dc065b20 msgr2=0x7fb6dc067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:25.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.252+0000 7fb6ff510640 1 --2- 172.21.15.131:0/214370938 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb6dc065b20 0x7fb6dc067fe0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0001060 tx=0x7fb6f00067b0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.252+0000 7fb6ff510640 1 -- 172.21.15.131:0/214370938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6f80ae690 msgr2=0x7fb6f80aeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:25.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.252+0000 7fb6ff510640 1 --2- 172.21.15.131:0/214370938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6f80ae690 0x7fb6f80aeb20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4009ee0 tx=0x7fb6f4009780 comp rx=0 tx=0).stop 2023-09-12T14:55:25.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.252+0000 7fb6ff510640 1 -- 172.21.15.131:0/214370938 shutdown_connections 2023-09-12T14:55:25.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.252+0000 7fb6ff510640 1 --2- 172.21.15.131:0/214370938 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb6dc065b20 0x7fb6dc067fe0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.252+0000 7fb6ff510640 1 --2- 172.21.15.131:0/214370938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6f80ae690 0x7fb6f80aeb20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.252+0000 7fb6ff510640 1 --2- 172.21.15.131:0/214370938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6f814d700 0x7fb6f8140440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.252+0000 7fb6ff510640 1 --2- 172.21.15.131:0/214370938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6e4004d10 0x7fb6f8150940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.252+0000 7fb6ff510640 1 -- 172.21.15.131:0/214370938 >> 172.21.15.131:0/214370938 conn(0x7fb6f8069840 msgr2=0x7fb6f80b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:25.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.252+0000 7fb6ff510640 1 -- 172.21.15.131:0/214370938 shutdown_connections 2023-09-12T14:55:25.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.252+0000 7fb6ff510640 1 -- 172.21.15.131:0/214370938 wait complete. 2023-09-12T14:55:25.270 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T14:55:25.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.384+0000 7f210c71b640 1 -- 172.21.15.131:0/2016612717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21041574d0 msgr2=0x7f21040cacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:25.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.384+0000 7f210c71b640 1 --2- 172.21.15.131:0/2016612717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21041574d0 0x7f21040cacd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f21040cd990 tx=0x7f210001c870 comp rx=0 tx=0).stop 2023-09-12T14:55:25.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.384+0000 7f210c71b640 1 -- 172.21.15.131:0/2016612717 shutdown_connections 2023-09-12T14:55:25.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.384+0000 7f210c71b640 1 --2- 172.21.15.131:0/2016612717 >> v2:172.21.15.187:3300/0 conn(0x7f21040b2600 0x7f21040c9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.384+0000 7f210c71b640 1 --2- 172.21.15.131:0/2016612717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21041574d0 0x7f21040cacd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.384+0000 7f210c71b640 1 -- 172.21.15.131:0/2016612717 >> 172.21.15.131:0/2016612717 conn(0x7f2104069840 msgr2=0x7f21040cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:25.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.384+0000 7f210c71b640 1 -- 172.21.15.131:0/2016612717 shutdown_connections 2023-09-12T14:55:25.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.384+0000 7f210c71b640 1 -- 172.21.15.131:0/2016612717 wait complete. 2023-09-12T14:55:25.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.384+0000 7f210c71b640 1 Processor -- start 2023-09-12T14:55:25.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.384+0000 7f210c71b640 1 -- start start 2023-09-12T14:55:25.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f210c71b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21040b2600 0x7f2104149650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:25.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f210c71b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21040c98d0 0x7f210414d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:25.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f210c71b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21041574d0 0x7f21041579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:25.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f210c71b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2104149b90 con 0x7f21040c98d0 2023-09-12T14:55:25.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f210c71b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210414d970 con 0x7f21040b2600 2023-09-12T14:55:25.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f210c71b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210414b970 con 0x7f21041574d0 2023-09-12T14:55:25.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f2109cb6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21040c98d0 0x7f210414d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:25.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f2109cb6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21040c98d0 0x7f210414d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55940/0 (socket says 172.21.15.131:55940) 2023-09-12T14:55:25.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f2109cb6640 1 -- 172.21.15.131:0/3182890968 learned_addr learned my addr 172.21.15.131:0/3182890968 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:25.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f210acb8640 1 --2- 172.21.15.131:0/3182890968 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21041574d0 0x7f21041579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:25.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f2109cb6640 1 -- 172.21.15.131:0/3182890968 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21041574d0 msgr2=0x7f21041579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:25.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f210a4b7640 1 --2- 172.21.15.131:0/3182890968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21040b2600 0x7f2104149650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:25.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f2109cb6640 1 --2- 172.21.15.131:0/3182890968 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21041574d0 0x7f21041579a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f2109cb6640 1 -- 172.21.15.131:0/3182890968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21040b2600 msgr2=0x7f2104149650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:25.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f2109cb6640 1 --2- 172.21.15.131:0/3182890968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21040b2600 0x7f2104149650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f2109cb6640 1 -- 172.21.15.131:0/3182890968 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2100009560 con 0x7f21040c98d0 2023-09-12T14:55:25.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f210a4b7640 1 --2- 172.21.15.131:0/3182890968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21040b2600 0x7f2104149650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:25.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f2109cb6640 1 --2- 172.21.15.131:0/3182890968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21040c98d0 0x7f210414d430 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f20f400d950 tx=0x7f20f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:25.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f20faffd640 1 -- 172.21.15.131:0/3182890968 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f4009c30 con 0x7f21040c98d0 2023-09-12T14:55:25.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f20faffd640 1 -- 172.21.15.131:0/3182890968 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f20f4009e80 con 0x7f21040c98d0 2023-09-12T14:55:25.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f210c71b640 1 -- 172.21.15.131:0/3182890968 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f210414dbd0 con 0x7f21040c98d0 2023-09-12T14:55:25.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f20faffd640 1 -- 172.21.15.131:0/3182890968 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f40106c0 con 0x7f21040c98d0 2023-09-12T14:55:25.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f210c71b640 1 -- 172.21.15.131:0/3182890968 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f210414e190 con 0x7f21040c98d0 2023-09-12T14:55:25.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f210c71b640 1 -- 172.21.15.131:0/3182890968 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2104073580 con 0x7f21040c98d0 2023-09-12T14:55:25.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f20faffd640 1 -- 172.21.15.131:0/3182890968 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f20f4010820 con 0x7f21040c98d0 2023-09-12T14:55:25.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f20faffd640 1 --2- 172.21.15.131:0/3182890968 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f20e8065b50 0x7f20e8068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:25.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f210a4b7640 1 --2- 172.21.15.131:0/3182890968 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f20e8065b50 0x7f20e8068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:25.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f20faffd640 1 -- 172.21.15.131:0/3182890968 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20f4083740 con 0x7f21040c98d0 2023-09-12T14:55:25.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.388+0000 7f210a4b7640 1 --2- 172.21.15.131:0/3182890968 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f20e8065b50 0x7f20e8068010 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f21000af000 tx=0x7f210001b0b0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:25.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.392+0000 7f20faffd640 1 -- 172.21.15.131:0/3182890968 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f20f4051090 con 0x7f21040c98d0 2023-09-12T14:55:25.456 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-12T14:55:25.458 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-12T14:55:25.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.536+0000 7f210c71b640 1 -- 172.21.15.131:0/3182890968 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f21040ac140 con 0x7f21040c98d0 2023-09-12T14:55:25.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.536+0000 7f20faffd640 1 -- 172.21.15.131:0/3182890968 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f20f4020020 con 0x7f21040c98d0 2023-09-12T14:55:25.539 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-09-12T14:55:25.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.536+0000 7f210c71b640 1 -- 172.21.15.131:0/3182890968 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f20e8065b50 msgr2=0x7f20e8068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:25.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.536+0000 7f210c71b640 1 --2- 172.21.15.131:0/3182890968 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f20e8065b50 0x7f20e8068010 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f21000af000 tx=0x7f210001b0b0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.536+0000 7f210c71b640 1 -- 172.21.15.131:0/3182890968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21040c98d0 msgr2=0x7f210414d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:25.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.536+0000 7f210c71b640 1 --2- 172.21.15.131:0/3182890968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21040c98d0 0x7f210414d430 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f20f400d950 tx=0x7f20f400de10 comp rx=0 tx=0).stop 2023-09-12T14:55:25.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.536+0000 7f210c71b640 1 -- 172.21.15.131:0/3182890968 shutdown_connections 2023-09-12T14:55:25.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.536+0000 7f210c71b640 1 --2- 172.21.15.131:0/3182890968 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f20e8065b50 0x7f20e8068010 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.536+0000 7f210c71b640 1 --2- 172.21.15.131:0/3182890968 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21041574d0 0x7f21041579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.536+0000 7f210c71b640 1 --2- 172.21.15.131:0/3182890968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21040c98d0 0x7f210414d430 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.536+0000 7f210c71b640 1 --2- 172.21.15.131:0/3182890968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21040b2600 0x7f2104149650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.536+0000 7f210c71b640 1 -- 172.21.15.131:0/3182890968 >> 172.21.15.131:0/3182890968 conn(0x7f2104069840 msgr2=0x7f210406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:25.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.536+0000 7f210c71b640 1 -- 172.21.15.131:0/3182890968 shutdown_connections 2023-09-12T14:55:25.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.536+0000 7f210c71b640 1 -- 172.21.15.131:0/3182890968 wait complete. 2023-09-12T14:55:25.556 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T14:55:25.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b320d2640 1 -- 172.21.15.131:0/3478526485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b2c0abb40 msgr2=0x7f3b2c06b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:25.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b320d2640 1 --2- 172.21.15.131:0/3478526485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b2c0abb40 0x7f3b2c06b4a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3b20009840 tx=0x7f3b2001c790 comp rx=0 tx=0).stop 2023-09-12T14:55:25.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b320d2640 1 -- 172.21.15.131:0/3478526485 shutdown_connections 2023-09-12T14:55:25.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b320d2640 1 --2- 172.21.15.131:0/3478526485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b2c0abb40 0x7f3b2c06b4a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b320d2640 1 -- 172.21.15.131:0/3478526485 >> 172.21.15.131:0/3478526485 conn(0x7f3b2c069840 msgr2=0x7f3b2c0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:25.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b320d2640 1 -- 172.21.15.131:0/3478526485 shutdown_connections 2023-09-12T14:55:25.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b320d2640 1 -- 172.21.15.131:0/3478526485 wait complete. 2023-09-12T14:55:25.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b320d2640 1 Processor -- start 2023-09-12T14:55:25.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b320d2640 1 -- start start 2023-09-12T14:55:25.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b320d2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b2c0abb40 0x7f3b2c15a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:25.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b320d2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b2c0c9370 0x7f3b2c14a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:25.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b320d2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b2c156930 0x7f3b2c148d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:25.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b320d2640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c1492b0 con 0x7f3b2c0abb40 2023-09-12T14:55:25.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b320d2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c1494e0 con 0x7f3b2c156930 2023-09-12T14:55:25.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b320d2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c1497c0 con 0x7f3b2c0c9370 2023-09-12T14:55:25.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b2b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b2c0abb40 0x7f3b2c15a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:25.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b2b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b2c0abb40 0x7f3b2c15a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55954/0 (socket says 172.21.15.131:55954) 2023-09-12T14:55:25.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b2b7fe640 1 -- 172.21.15.131:0/944807188 learned_addr learned my addr 172.21.15.131:0/944807188 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:25.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b2affd640 1 --2- 172.21.15.131:0/944807188 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b2c0c9370 0x7f3b2c14a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:25.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b2bfff640 1 --2- 172.21.15.131:0/944807188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b2c156930 0x7f3b2c148d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:25.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b2b7fe640 1 -- 172.21.15.131:0/944807188 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b2c0c9370 msgr2=0x7f3b2c14a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:25.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b2b7fe640 1 --2- 172.21.15.131:0/944807188 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b2c0c9370 0x7f3b2c14a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b2b7fe640 1 -- 172.21.15.131:0/944807188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b2c156930 msgr2=0x7f3b2c148d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:25.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b2b7fe640 1 --2- 172.21.15.131:0/944807188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b2c156930 0x7f3b2c148d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b2b7fe640 1 -- 172.21.15.131:0/944807188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b200094a0 con 0x7f3b2c0abb40 2023-09-12T14:55:25.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b2b7fe640 1 --2- 172.21.15.131:0/944807188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b2c0abb40 0x7f3b2c15a2e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00c3c0 tx=0x7f3b1c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:25.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b13fff640 1 -- 172.21.15.131:0/944807188 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c011790 con 0x7f3b2c0abb40 2023-09-12T14:55:25.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b13fff640 1 -- 172.21.15.131:0/944807188 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3b1c0119e0 con 0x7f3b2c0abb40 2023-09-12T14:55:25.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b320d2640 1 -- 172.21.15.131:0/944807188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b2c149aa0 con 0x7f3b2c0abb40 2023-09-12T14:55:25.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b13fff640 1 -- 172.21.15.131:0/944807188 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c011e30 con 0x7f3b2c0abb40 2023-09-12T14:55:25.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.680+0000 7f3b320d2640 1 -- 172.21.15.131:0/944807188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b2c14a060 con 0x7f3b2c0abb40 2023-09-12T14:55:25.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.684+0000 7f3b320d2640 1 -- 172.21.15.131:0/944807188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b2c14a250 con 0x7f3b2c0abb40 2023-09-12T14:55:25.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.684+0000 7f3b13fff640 1 -- 172.21.15.131:0/944807188 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f3b1c01c460 con 0x7f3b2c0abb40 2023-09-12T14:55:25.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.684+0000 7f3b13fff640 1 --2- 172.21.15.131:0/944807188 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3b18065b20 0x7f3b18067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:25.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.684+0000 7f3b13fff640 1 -- 172.21.15.131:0/944807188 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b1c0832d0 con 0x7f3b2c0abb40 2023-09-12T14:55:25.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.684+0000 7f3b2affd640 1 --2- 172.21.15.131:0/944807188 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3b18065b20 0x7f3b18067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:25.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.684+0000 7f3b2affd640 1 --2- 172.21.15.131:0/944807188 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3b18065b20 0x7f3b18067fe0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3b200af000 tx=0x7f3b2001afd0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:25.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.688+0000 7f3b13fff640 1 -- 172.21.15.131:0/944807188 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f3b1c050cb0 con 0x7f3b2c0abb40 2023-09-12T14:55:25.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.828+0000 7f3b320d2640 1 -- 172.21.15.131:0/944807188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3b2c0b1bc0 con 0x7f3b2c0abb40 2023-09-12T14:55:25.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.828+0000 7f3b13fff640 1 -- 172.21.15.131:0/944807188 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3b1c0542d0 con 0x7f3b2c0abb40 2023-09-12T14:55:25.832 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-09-12T14:55:25.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.828+0000 7f3b320d2640 1 -- 172.21.15.131:0/944807188 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3b18065b20 msgr2=0x7f3b18067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:25.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.828+0000 7f3b320d2640 1 --2- 172.21.15.131:0/944807188 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3b18065b20 0x7f3b18067fe0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3b200af000 tx=0x7f3b2001afd0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.828+0000 7f3b320d2640 1 -- 172.21.15.131:0/944807188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b2c0abb40 msgr2=0x7f3b2c15a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:25.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.828+0000 7f3b320d2640 1 --2- 172.21.15.131:0/944807188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b2c0abb40 0x7f3b2c15a2e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00c3c0 tx=0x7f3b1c00c880 comp rx=0 tx=0).stop 2023-09-12T14:55:25.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.828+0000 7f3b320d2640 1 -- 172.21.15.131:0/944807188 shutdown_connections 2023-09-12T14:55:25.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.828+0000 7f3b320d2640 1 --2- 172.21.15.131:0/944807188 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3b18065b20 0x7f3b18067fe0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.828+0000 7f3b320d2640 1 --2- 172.21.15.131:0/944807188 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b2c0c9370 0x7f3b2c14a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.828+0000 7f3b320d2640 1 --2- 172.21.15.131:0/944807188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b2c156930 0x7f3b2c148d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.828+0000 7f3b320d2640 1 --2- 172.21.15.131:0/944807188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b2c0abb40 0x7f3b2c15a2e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.828+0000 7f3b320d2640 1 -- 172.21.15.131:0/944807188 >> 172.21.15.131:0/944807188 conn(0x7f3b2c069840 msgr2=0x7f3b2c0cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:25.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.828+0000 7f3b320d2640 1 -- 172.21.15.131:0/944807188 shutdown_connections 2023-09-12T14:55:25.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.828+0000 7f3b320d2640 1 -- 172.21.15.131:0/944807188 wait complete. 2023-09-12T14:55:25.848 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:55:25.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fde441640 1 -- 172.21.15.131:0/2356268731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fd80704d0 msgr2=0x7f9fd806bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:25.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fde441640 1 --2- 172.21.15.131:0/2356268731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fd80704d0 0x7f9fd806bf70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8070a30 tx=0x7f9fc001c7e0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fde441640 1 -- 172.21.15.131:0/2356268731 shutdown_connections 2023-09-12T14:55:25.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fde441640 1 --2- 172.21.15.131:0/2356268731 >> v2:172.21.15.187:3300/0 conn(0x7f9fd806c4b0 0x7f9fd80cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fde441640 1 --2- 172.21.15.131:0/2356268731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fd80704d0 0x7f9fd806bf70 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fde441640 1 -- 172.21.15.131:0/2356268731 >> 172.21.15.131:0/2356268731 conn(0x7f9fd80ae580 msgr2=0x7f9fd80ae990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:25.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fde441640 1 -- 172.21.15.131:0/2356268731 shutdown_connections 2023-09-12T14:55:25.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fde441640 1 -- 172.21.15.131:0/2356268731 wait complete. 2023-09-12T14:55:25.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fde441640 1 Processor -- start 2023-09-12T14:55:25.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fde441640 1 -- start start 2023-09-12T14:55:25.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fde441640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fd806c4b0 0x7f9fd80c4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:25.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fde441640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fd8157d20 0x7f9fd81492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:25.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fde441640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fd81546b0 0x7f9fd8153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:25.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fde441640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd80c69c0 con 0x7f9fd81546b0 2023-09-12T14:55:25.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fde441640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd80c5130 con 0x7f9fd8157d20 2023-09-12T14:55:25.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fde441640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd8152e50 con 0x7f9fd806c4b0 2023-09-12T14:55:25.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fdc9de640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fd81546b0 0x7f9fd8153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:25.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fdc9de640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fd81546b0 0x7f9fd8153930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55966/0 (socket says 172.21.15.131:55966) 2023-09-12T14:55:25.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fdc9de640 1 -- 172.21.15.131:0/2430318296 learned_addr learned my addr 172.21.15.131:0/2430318296 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:25.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fdc9de640 1 -- 172.21.15.131:0/2430318296 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fd806c4b0 msgr2=0x7f9fd80c4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:25.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fdc9de640 1 --2- 172.21.15.131:0/2430318296 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fd806c4b0 0x7f9fd80c4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fdc9de640 1 -- 172.21.15.131:0/2430318296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fd8157d20 msgr2=0x7f9fd81492a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:25.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fdc9de640 1 --2- 172.21.15.131:0/2430318296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fd8157d20 0x7f9fd81492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:25.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fdc9de640 1 -- 172.21.15.131:0/2430318296 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc00094a0 con 0x7f9fd81546b0 2023-09-12T14:55:25.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fdc9de640 1 --2- 172.21.15.131:0/2430318296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fd81546b0 0x7f9fd8153930 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9fd000b970 tx=0x7f9fd000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:25.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fd57fa640 1 -- 172.21.15.131:0/2430318296 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd000cb40 con 0x7f9fd81546b0 2023-09-12T14:55:25.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fd57fa640 1 -- 172.21.15.131:0/2430318296 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9fd000cdb0 con 0x7f9fd81546b0 2023-09-12T14:55:25.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fde441640 1 -- 172.21.15.131:0/2430318296 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd8153130 con 0x7f9fd81546b0 2023-09-12T14:55:25.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fd57fa640 1 -- 172.21.15.131:0/2430318296 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd00146a0 con 0x7f9fd81546b0 2023-09-12T14:55:25.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.956+0000 7f9fde441640 1 -- 172.21.15.131:0/2430318296 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd81499b0 con 0x7f9fd81546b0 2023-09-12T14:55:25.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.960+0000 7f9fde441640 1 -- 172.21.15.131:0/2430318296 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd8073580 con 0x7f9fd81546b0 2023-09-12T14:55:25.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.960+0000 7f9fd57fa640 1 -- 172.21.15.131:0/2430318296 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f9fd001d3b0 con 0x7f9fd81546b0 2023-09-12T14:55:25.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.960+0000 7f9fd57fa640 1 --2- 172.21.15.131:0/2430318296 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9fb8065b50 0x7f9fb8068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:25.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.960+0000 7f9fd7fff640 1 --2- 172.21.15.131:0/2430318296 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9fb8065b50 0x7f9fb8068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:25.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.960+0000 7f9fd57fa640 1 -- 172.21.15.131:0/2430318296 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9fd00834b0 con 0x7f9fd81546b0 2023-09-12T14:55:25.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.960+0000 7f9fd7fff640 1 --2- 172.21.15.131:0/2430318296 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9fb8065b50 0x7f9fb8068010 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80046c0 tx=0x7f9fc8007a80 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:25.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:25.964+0000 7f9fd57fa640 1 -- 172.21.15.131:0/2430318296 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f9fd0050e00 con 0x7f9fd81546b0 2023-09-12T14:55:26.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.120+0000 7f9fde441640 1 -- 172.21.15.131:0/2430318296 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fd80cc530 con 0x7f9fd81546b0 2023-09-12T14:55:26.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.120+0000 7f9fd57fa640 1 -- 172.21.15.131:0/2430318296 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f9fd0054420 con 0x7f9fd81546b0 2023-09-12T14:55:26.125 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:26.125 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:54:59.694778+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:54:55.689983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T14:54:57.029447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:55:26.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.124+0000 7f9fde441640 1 -- 172.21.15.131:0/2430318296 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9fb8065b50 msgr2=0x7f9fb8068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:26.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.124+0000 7f9fde441640 1 --2- 172.21.15.131:0/2430318296 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9fb8065b50 0x7f9fb8068010 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80046c0 tx=0x7f9fc8007a80 comp rx=0 tx=0).stop 2023-09-12T14:55:26.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.124+0000 7f9fde441640 1 -- 172.21.15.131:0/2430318296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fd81546b0 msgr2=0x7f9fd8153930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:26.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.124+0000 7f9fde441640 1 --2- 172.21.15.131:0/2430318296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fd81546b0 0x7f9fd8153930 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9fd000b970 tx=0x7f9fd000be30 comp rx=0 tx=0).stop 2023-09-12T14:55:26.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.124+0000 7f9fde441640 1 -- 172.21.15.131:0/2430318296 shutdown_connections 2023-09-12T14:55:26.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.124+0000 7f9fde441640 1 --2- 172.21.15.131:0/2430318296 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9fb8065b50 0x7f9fb8068010 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.124+0000 7f9fde441640 1 --2- 172.21.15.131:0/2430318296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fd81546b0 0x7f9fd8153930 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.124+0000 7f9fde441640 1 --2- 172.21.15.131:0/2430318296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fd8157d20 0x7f9fd81492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.124+0000 7f9fde441640 1 --2- 172.21.15.131:0/2430318296 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fd806c4b0 0x7f9fd80c4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.124+0000 7f9fde441640 1 -- 172.21.15.131:0/2430318296 >> 172.21.15.131:0/2430318296 conn(0x7f9fd80ae580 msgr2=0x7f9fd8070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:26.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.124+0000 7f9fde441640 1 -- 172.21.15.131:0/2430318296 shutdown_connections 2023-09-12T14:55:26.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.124+0000 7f9fde441640 1 -- 172.21.15.131:0/2430318296 wait complete. 2023-09-12T14:55:26.144 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T14:54:45.654004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T14:54:55.689983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T14:54:57.029447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-12T14:55:26.144 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T14:55:26.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.268+0000 7fbe2fd4c640 1 -- 172.21.15.131:0/264434866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe2806f390 msgr2=0x7fbe2806f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:26.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.268+0000 7fbe2fd4c640 1 --2- 172.21.15.131:0/264434866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe2806f390 0x7fbe2806f820 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbe2806c710 tx=0x7fbe2401cd30 comp rx=0 tx=0).stop 2023-09-12T14:55:26.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.268+0000 7fbe2fd4c640 1 -- 172.21.15.131:0/264434866 shutdown_connections 2023-09-12T14:55:26.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.268+0000 7fbe2fd4c640 1 --2- 172.21.15.131:0/264434866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe2806f390 0x7fbe2806f820 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.268+0000 7fbe2fd4c640 1 --2- 172.21.15.131:0/264434866 >> v2:172.21.15.187:3300/0 conn(0x7fbe28158180 0x7fbe280cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.268+0000 7fbe2fd4c640 1 --2- 172.21.15.131:0/264434866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe280ab730 0x7fbe280abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.268+0000 7fbe2fd4c640 1 -- 172.21.15.131:0/264434866 >> 172.21.15.131:0/264434866 conn(0x7fbe28069840 msgr2=0x7fbe280cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:26.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.268+0000 7fbe2fd4c640 1 -- 172.21.15.131:0/264434866 shutdown_connections 2023-09-12T14:55:26.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.268+0000 7fbe2fd4c640 1 -- 172.21.15.131:0/264434866 wait complete. 2023-09-12T14:55:26.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe2fd4c640 1 Processor -- start 2023-09-12T14:55:26.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe2fd4c640 1 -- start start 2023-09-12T14:55:26.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe2fd4c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe2806f390 0x7fbe2814fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:26.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe2fd4c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe280ab730 0x7fbe2814b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:26.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe2fd4c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe28158180 0x7fbe281f5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:26.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe2fd4c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe281500f0 con 0x7fbe28158180 2023-09-12T14:55:26.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe2fd4c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe280ab490 con 0x7fbe2806f390 2023-09-12T14:55:26.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe2fd4c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe281597e0 con 0x7fbe280ab730 2023-09-12T14:55:26.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe2e2e9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe28158180 0x7fbe281f5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:26.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe2e2e9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe28158180 0x7fbe281f5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55970/0 (socket says 172.21.15.131:55970) 2023-09-12T14:55:26.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe2e2e9640 1 -- 172.21.15.131:0/3731584522 learned_addr learned my addr 172.21.15.131:0/3731584522 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:26.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe2d2e7640 1 --2- 172.21.15.131:0/3731584522 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe280ab730 0x7fbe2814b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:26.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe2dae8640 1 --2- 172.21.15.131:0/3731584522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe2806f390 0x7fbe2814fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:26.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe2e2e9640 1 -- 172.21.15.131:0/3731584522 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe280ab730 msgr2=0x7fbe2814b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:26.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe2e2e9640 1 --2- 172.21.15.131:0/3731584522 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe280ab730 0x7fbe2814b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe2e2e9640 1 -- 172.21.15.131:0/3731584522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe2806f390 msgr2=0x7fbe2814fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:26.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe2e2e9640 1 --2- 172.21.15.131:0/3731584522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe2806f390 0x7fbe2814fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe2e2e9640 1 -- 172.21.15.131:0/3731584522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe240098e0 con 0x7fbe28158180 2023-09-12T14:55:26.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe2d2e7640 1 --2- 172.21.15.131:0/3731584522 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe280ab730 0x7fbe2814b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:26.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe2e2e9640 1 --2- 172.21.15.131:0/3731584522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe28158180 0x7fbe281f5450 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbe2401c7a0 tx=0x7fbe24004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:26.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe1e7fc640 1 -- 172.21.15.131:0/3731584522 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe240bb070 con 0x7fbe28158180 2023-09-12T14:55:26.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe1e7fc640 1 -- 172.21.15.131:0/3731584522 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbe24003c80 con 0x7fbe28158180 2023-09-12T14:55:26.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe1e7fc640 1 -- 172.21.15.131:0/3731584522 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe240ad900 con 0x7fbe28158180 2023-09-12T14:55:26.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe2fd4c640 1 -- 172.21.15.131:0/3731584522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe28159ac0 con 0x7fbe28158180 2023-09-12T14:55:26.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe2fd4c640 1 -- 172.21.15.131:0/3731584522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe2815a080 con 0x7fbe28158180 2023-09-12T14:55:26.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe2fd4c640 1 -- 172.21.15.131:0/3731584522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe28073580 con 0x7fbe28158180 2023-09-12T14:55:26.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe1e7fc640 1 -- 172.21.15.131:0/3731584522 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fbe240b4040 con 0x7fbe28158180 2023-09-12T14:55:26.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe1e7fc640 1 --2- 172.21.15.131:0/3731584522 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fbe0c065b20 0x7fbe0c067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:26.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe1e7fc640 1 -- 172.21.15.131:0/3731584522 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe2411de70 con 0x7fbe28158180 2023-09-12T14:55:26.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.272+0000 7fbe2dae8640 1 --2- 172.21.15.131:0/3731584522 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fbe0c065b20 0x7fbe0c067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:26.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.276+0000 7fbe2dae8640 1 --2- 172.21.15.131:0/3731584522 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fbe0c065b20 0x7fbe0c067fe0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbe200058d0 tx=0x7fbe20005c70 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:26.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.276+0000 7fbe1e7fc640 1 -- 172.21.15.131:0/3731584522 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fbe240eb8e0 con 0x7fbe28158180 2023-09-12T14:55:26.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.420+0000 7fbe2fd4c640 1 -- 172.21.15.131:0/3731584522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbe28152560 con 0x7fbe28158180 2023-09-12T14:55:26.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.420+0000 7fbe1e7fc640 1 -- 172.21.15.131:0/3731584522 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbe240b2020 con 0x7fbe28158180 2023-09-12T14:55:26.423 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-09-12T14:55:26.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.420+0000 7fbe2fd4c640 1 -- 172.21.15.131:0/3731584522 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fbe0c065b20 msgr2=0x7fbe0c067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:26.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.420+0000 7fbe2fd4c640 1 --2- 172.21.15.131:0/3731584522 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fbe0c065b20 0x7fbe0c067fe0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbe200058d0 tx=0x7fbe20005c70 comp rx=0 tx=0).stop 2023-09-12T14:55:26.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.420+0000 7fbe2fd4c640 1 -- 172.21.15.131:0/3731584522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe28158180 msgr2=0x7fbe281f5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:26.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.420+0000 7fbe2fd4c640 1 --2- 172.21.15.131:0/3731584522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe28158180 0x7fbe281f5450 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbe2401c7a0 tx=0x7fbe24004dc0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.420+0000 7fbe2fd4c640 1 -- 172.21.15.131:0/3731584522 shutdown_connections 2023-09-12T14:55:26.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.420+0000 7fbe2fd4c640 1 --2- 172.21.15.131:0/3731584522 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fbe0c065b20 0x7fbe0c067fe0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.420+0000 7fbe2fd4c640 1 --2- 172.21.15.131:0/3731584522 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe280ab730 0x7fbe2814b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.420+0000 7fbe2fd4c640 1 --2- 172.21.15.131:0/3731584522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe28158180 0x7fbe281f5450 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.420+0000 7fbe2fd4c640 1 --2- 172.21.15.131:0/3731584522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe2806f390 0x7fbe2814fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.420+0000 7fbe2fd4c640 1 -- 172.21.15.131:0/3731584522 >> 172.21.15.131:0/3731584522 conn(0x7fbe28069840 msgr2=0x7fbe280addc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:26.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.420+0000 7fbe2fd4c640 1 -- 172.21.15.131:0/3731584522 shutdown_connections 2023-09-12T14:55:26.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.420+0000 7fbe2fd4c640 1 -- 172.21.15.131:0/3731584522 wait complete. 2023-09-12T14:55:26.440 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T14:55:26.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.552+0000 7f269b68f640 1 -- 172.21.15.131:0/1982111402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2694158180 msgr2=0x7f26940cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:26.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.552+0000 7f269b68f640 1 --2- 172.21.15.131:0/1982111402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2694158180 0x7f26940cae70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f269406eea0 tx=0x7f268801c6d0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.552+0000 7f269b68f640 1 -- 172.21.15.131:0/1982111402 shutdown_connections 2023-09-12T14:55:26.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.552+0000 7f269b68f640 1 --2- 172.21.15.131:0/1982111402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2694158180 0x7f26940cae70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.552+0000 7f269b68f640 1 --2- 172.21.15.131:0/1982111402 >> v2:172.21.15.187:3300/0 conn(0x7f26940ab730 0x7f26940abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.552+0000 7f269b68f640 1 -- 172.21.15.131:0/1982111402 >> 172.21.15.131:0/1982111402 conn(0x7f2694069840 msgr2=0x7f26940cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:26.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.552+0000 7f269b68f640 1 -- 172.21.15.131:0/1982111402 shutdown_connections 2023-09-12T14:55:26.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.552+0000 7f269b68f640 1 -- 172.21.15.131:0/1982111402 wait complete. 2023-09-12T14:55:26.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.556+0000 7f269b68f640 1 Processor -- start 2023-09-12T14:55:26.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.556+0000 7f269b68f640 1 -- start start 2023-09-12T14:55:26.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.556+0000 7f269b68f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f269406f390 0x7f26940aca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:26.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.556+0000 7f269b68f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26940ab730 0x7f269413bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:26.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.556+0000 7f269b68f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2694158180 0x7f269413c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:26.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.556+0000 7f269b68f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26940b0fb0 con 0x7f2694158180 2023-09-12T14:55:26.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.556+0000 7f269b68f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26940b0420 con 0x7f269406f390 2023-09-12T14:55:26.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.556+0000 7f269b68f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26940b1200 con 0x7f26940ab730 2023-09-12T14:55:26.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.556+0000 7f2699c2c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2694158180 0x7f269413c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:26.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.556+0000 7f2699c2c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2694158180 0x7f269413c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55986/0 (socket says 172.21.15.131:55986) 2023-09-12T14:55:26.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.556+0000 7f2699c2c640 1 -- 172.21.15.131:0/2626373548 learned_addr learned my addr 172.21.15.131:0/2626373548 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:26.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.556+0000 7f269942b640 1 --2- 172.21.15.131:0/2626373548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f269406f390 0x7f26940aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:26.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.556+0000 7f2698c2a640 1 --2- 172.21.15.131:0/2626373548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26940ab730 0x7f269413bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:26.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.556+0000 7f2699c2c640 1 -- 172.21.15.131:0/2626373548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26940ab730 msgr2=0x7f269413bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:26.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.556+0000 7f2699c2c640 1 --2- 172.21.15.131:0/2626373548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26940ab730 0x7f269413bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.556+0000 7f2699c2c640 1 -- 172.21.15.131:0/2626373548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f269406f390 msgr2=0x7f26940aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:26.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.556+0000 7f2699c2c640 1 --2- 172.21.15.131:0/2626373548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f269406f390 0x7f26940aca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.556+0000 7f2699c2c640 1 -- 172.21.15.131:0/2626373548 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26880094a0 con 0x7f2694158180 2023-09-12T14:55:26.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.556+0000 7f269942b640 1 --2- 172.21.15.131:0/2626373548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f269406f390 0x7f26940aca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:26.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.556+0000 7f2699c2c640 1 --2- 172.21.15.131:0/2626373548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2694158180 0x7f269413c410 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f268400c750 tx=0x7f268400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:26.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.556+0000 7f2691ffb640 1 -- 172.21.15.131:0/2626373548 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268400a840 con 0x7f2694158180 2023-09-12T14:55:26.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.556+0000 7f2691ffb640 1 -- 172.21.15.131:0/2626373548 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f268400aab0 con 0x7f2694158180 2023-09-12T14:55:26.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.556+0000 7f269b68f640 1 -- 172.21.15.131:0/2626373548 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26940b14e0 con 0x7f2694158180 2023-09-12T14:55:26.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.556+0000 7f2691ffb640 1 -- 172.21.15.131:0/2626373548 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684012420 con 0x7f2694158180 2023-09-12T14:55:26.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.556+0000 7f269b68f640 1 -- 172.21.15.131:0/2626373548 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26940b1aa0 con 0x7f2694158180 2023-09-12T14:55:26.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.556+0000 7f269b68f640 1 -- 172.21.15.131:0/2626373548 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f269414b010 con 0x7f2694158180 2023-09-12T14:55:26.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.556+0000 7f2691ffb640 1 -- 172.21.15.131:0/2626373548 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f2684012580 con 0x7f2694158180 2023-09-12T14:55:26.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.560+0000 7f2691ffb640 1 --2- 172.21.15.131:0/2626373548 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2680065b50 0x7f2680068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:26.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.560+0000 7f2691ffb640 1 -- 172.21.15.131:0/2626373548 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2684083910 con 0x7f2694158180 2023-09-12T14:55:26.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.560+0000 7f269942b640 1 --2- 172.21.15.131:0/2626373548 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2680065b50 0x7f2680068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:26.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.560+0000 7f269942b640 1 --2- 172.21.15.131:0/2626373548 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2680065b50 0x7f2680068010 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f268c009e30 tx=0x7f268c0093c0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:26.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.564+0000 7f2691ffb640 1 -- 172.21.15.131:0/2626373548 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f269414b010 con 0x7f2694158180 2023-09-12T14:55:26.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.704+0000 7f269b68f640 1 -- 172.21.15.131:0/2626373548 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f269414b010 con 0x7f2694158180 2023-09-12T14:55:26.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.704+0000 7f2691ffb640 1 -- 172.21.15.131:0/2626373548 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2684016020 con 0x7f2694158180 2023-09-12T14:55:26.708 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-09-12T14:55:26.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.708+0000 7f269b68f640 1 -- 172.21.15.131:0/2626373548 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2680065b50 msgr2=0x7f2680068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:26.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.708+0000 7f269b68f640 1 --2- 172.21.15.131:0/2626373548 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2680065b50 0x7f2680068010 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f268c009e30 tx=0x7f268c0093c0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.708+0000 7f269b68f640 1 -- 172.21.15.131:0/2626373548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2694158180 msgr2=0x7f269413c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:26.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.708+0000 7f269b68f640 1 --2- 172.21.15.131:0/2626373548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2694158180 0x7f269413c410 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f268400c750 tx=0x7f268400cc10 comp rx=0 tx=0).stop 2023-09-12T14:55:26.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.708+0000 7f269b68f640 1 -- 172.21.15.131:0/2626373548 shutdown_connections 2023-09-12T14:55:26.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.708+0000 7f269b68f640 1 --2- 172.21.15.131:0/2626373548 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2680065b50 0x7f2680068010 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.708+0000 7f269b68f640 1 --2- 172.21.15.131:0/2626373548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26940ab730 0x7f269413bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.708+0000 7f269b68f640 1 --2- 172.21.15.131:0/2626373548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2694158180 0x7f269413c410 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.708+0000 7f269b68f640 1 --2- 172.21.15.131:0/2626373548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f269406f390 0x7f26940aca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.708+0000 7f269b68f640 1 -- 172.21.15.131:0/2626373548 >> 172.21.15.131:0/2626373548 conn(0x7f2694069840 msgr2=0x7f26940aa8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:26.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.708+0000 7f269b68f640 1 -- 172.21.15.131:0/2626373548 shutdown_connections 2023-09-12T14:55:26.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.708+0000 7f269b68f640 1 -- 172.21.15.131:0/2626373548 wait complete. 2023-09-12T14:55:26.726 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T14:55:26.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d5964e640 1 -- 172.21.15.131:0/4149374164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d540abc70 msgr2=0x7f3d5406f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:26.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d5964e640 1 --2- 172.21.15.131:0/4149374164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d540abc70 0x7f3d5406f390 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3d540cd9c0 tx=0x7f3d4c01cd30 comp rx=0 tx=0).stop 2023-09-12T14:55:26.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d5964e640 1 -- 172.21.15.131:0/4149374164 shutdown_connections 2023-09-12T14:55:26.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d5964e640 1 --2- 172.21.15.131:0/4149374164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d540abc70 0x7f3d5406f390 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d5964e640 1 --2- 172.21.15.131:0/4149374164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d540cb010 0x7f3d540ab730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d5964e640 1 --2- 172.21.15.131:0/4149374164 >> v2:172.21.15.187:3300/0 conn(0x7f3d541556f0 0x7f3d54158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d5964e640 1 -- 172.21.15.131:0/4149374164 >> 172.21.15.131:0/4149374164 conn(0x7f3d54069840 msgr2=0x7f3d540cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:26.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d5964e640 1 -- 172.21.15.131:0/4149374164 shutdown_connections 2023-09-12T14:55:26.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d5964e640 1 -- 172.21.15.131:0/4149374164 wait complete. 2023-09-12T14:55:26.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d5964e640 1 Processor -- start 2023-09-12T14:55:26.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d5964e640 1 -- start start 2023-09-12T14:55:26.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d5964e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d540abc70 0x7f3d541402a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:26.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d5964e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d540cb010 0x7f3d541f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:26.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d5964e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d541556f0 0x7f3d540aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:26.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d5964e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d54150b60 con 0x7f3d540cb010 2023-09-12T14:55:26.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d5964e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d541407e0 con 0x7f3d541556f0 2023-09-12T14:55:26.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d5964e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d54162830 con 0x7f3d540abc70 2023-09-12T14:55:26.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d527fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d540cb010 0x7f3d541f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:26.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d527fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d540cb010 0x7f3d541f3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56006/0 (socket says 172.21.15.131:56006) 2023-09-12T14:55:26.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d527fc640 1 -- 172.21.15.131:0/3833919948 learned_addr learned my addr 172.21.15.131:0/3833919948 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:26.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d527fc640 1 -- 172.21.15.131:0/3833919948 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d540abc70 msgr2=0x7f3d541402a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:26.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d527fc640 1 --2- 172.21.15.131:0/3833919948 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d540abc70 0x7f3d541402a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d527fc640 1 -- 172.21.15.131:0/3833919948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d541556f0 msgr2=0x7f3d540aa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:26.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d527fc640 1 --2- 172.21.15.131:0/3833919948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d541556f0 0x7f3d540aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d527fc640 1 -- 172.21.15.131:0/3833919948 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4c0098e0 con 0x7f3d540cb010 2023-09-12T14:55:26.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d527fc640 1 --2- 172.21.15.131:0/3833919948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d540cb010 0x7f3d541f3ca0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3d4800c940 tx=0x7f3d4800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:26.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d3b7fe640 1 -- 172.21.15.131:0/3833919948 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d48007bb0 con 0x7f3d540cb010 2023-09-12T14:55:26.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d3b7fe640 1 -- 172.21.15.131:0/3833919948 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3d48007e20 con 0x7f3d540cb010 2023-09-12T14:55:26.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d3b7fe640 1 -- 172.21.15.131:0/3833919948 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4800f6b0 con 0x7f3d540cb010 2023-09-12T14:55:26.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d5964e640 1 -- 172.21.15.131:0/3833919948 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d54162b10 con 0x7f3d540cb010 2023-09-12T14:55:26.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d5964e640 1 -- 172.21.15.131:0/3833919948 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d541630d0 con 0x7f3d540cb010 2023-09-12T14:55:26.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.840+0000 7f3d5964e640 1 -- 172.21.15.131:0/3833919948 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d541632c0 con 0x7f3d540cb010 2023-09-12T14:55:26.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.844+0000 7f3d3b7fe640 1 -- 172.21.15.131:0/3833919948 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f3d4800f810 con 0x7f3d540cb010 2023-09-12T14:55:26.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.844+0000 7f3d3b7fe640 1 --2- 172.21.15.131:0/3833919948 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3d40065b20 0x7f3d40067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:26.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.844+0000 7f3d52ffd640 1 --2- 172.21.15.131:0/3833919948 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3d40065b20 0x7f3d40067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:26.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.844+0000 7f3d3b7fe640 1 -- 172.21.15.131:0/3833919948 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d48083bd0 con 0x7f3d540cb010 2023-09-12T14:55:26.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.844+0000 7f3d52ffd640 1 --2- 172.21.15.131:0/3833919948 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3d40065b20 0x7f3d40067fe0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3d4400a860 tx=0x7f3d44005b60 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:26.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.848+0000 7f3d3b7fe640 1 -- 172.21.15.131:0/3833919948 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f3d48051510 con 0x7f3d540cb010 2023-09-12T14:55:26.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.992+0000 7f3d5964e640 1 -- 172.21.15.131:0/3833919948 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3d54073580 con 0x7f3d540cb010 2023-09-12T14:55:26.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.992+0000 7f3d3b7fe640 1 -- 172.21.15.131:0/3833919948 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3d4801d0a0 con 0x7f3d540cb010 2023-09-12T14:55:26.995 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-09-12T14:55:26.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.992+0000 7f3d5964e640 1 -- 172.21.15.131:0/3833919948 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3d40065b20 msgr2=0x7f3d40067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:26.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.992+0000 7f3d5964e640 1 --2- 172.21.15.131:0/3833919948 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3d40065b20 0x7f3d40067fe0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3d4400a860 tx=0x7f3d44005b60 comp rx=0 tx=0).stop 2023-09-12T14:55:26.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.992+0000 7f3d5964e640 1 -- 172.21.15.131:0/3833919948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d540cb010 msgr2=0x7f3d541f3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:26.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.992+0000 7f3d5964e640 1 --2- 172.21.15.131:0/3833919948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d540cb010 0x7f3d541f3ca0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3d4800c940 tx=0x7f3d4800ce00 comp rx=0 tx=0).stop 2023-09-12T14:55:26.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.992+0000 7f3d5964e640 1 -- 172.21.15.131:0/3833919948 shutdown_connections 2023-09-12T14:55:26.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.992+0000 7f3d5964e640 1 --2- 172.21.15.131:0/3833919948 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3d40065b20 0x7f3d40067fe0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.992+0000 7f3d5964e640 1 --2- 172.21.15.131:0/3833919948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d541556f0 0x7f3d540aa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.992+0000 7f3d5964e640 1 --2- 172.21.15.131:0/3833919948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d540cb010 0x7f3d541f3ca0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.992+0000 7f3d5964e640 1 --2- 172.21.15.131:0/3833919948 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d540abc70 0x7f3d541402a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:26.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.992+0000 7f3d5964e640 1 -- 172.21.15.131:0/3833919948 >> 172.21.15.131:0/3833919948 conn(0x7f3d54069840 msgr2=0x7f3d540adf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:26.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.992+0000 7f3d5964e640 1 -- 172.21.15.131:0/3833919948 shutdown_connections 2023-09-12T14:55:26.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:26.992+0000 7f3d5964e640 1 -- 172.21.15.131:0/3833919948 wait complete. 2023-09-12T14:55:27.012 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:55:27.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef9a61640 1 -- 172.21.15.131:0/116117841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ef40cf2d0 msgr2=0x7f0ef415e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:27.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef9a61640 1 --2- 172.21.15.131:0/116117841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ef40cf2d0 0x7f0ef415e2e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009930 tx=0x7f0ee401c880 comp rx=0 tx=0).stop 2023-09-12T14:55:27.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef9a61640 1 -- 172.21.15.131:0/116117841 shutdown_connections 2023-09-12T14:55:27.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef9a61640 1 --2- 172.21.15.131:0/116117841 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ef40d1680 0x7f0ef415e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:27.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef9a61640 1 --2- 172.21.15.131:0/116117841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ef40cf2d0 0x7f0ef415e2e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:27.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef9a61640 1 -- 172.21.15.131:0/116117841 >> 172.21.15.131:0/116117841 conn(0x7f0ef409a200 msgr2=0x7f0ef40d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:27.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef9a61640 1 -- 172.21.15.131:0/116117841 shutdown_connections 2023-09-12T14:55:27.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef9a61640 1 -- 172.21.15.131:0/116117841 wait complete. 2023-09-12T14:55:27.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef9a61640 1 Processor -- start 2023-09-12T14:55:27.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef9a61640 1 -- start start 2023-09-12T14:55:27.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef9a61640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ef40cf2d0 0x7f0ef40b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:27.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef9a61640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ef40cf6b0 0x7f0ef4155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:27.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef9a61640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ef40d1680 0x7f0ef415adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:27.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef9a61640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef415b800 con 0x7f0ef40d1680 2023-09-12T14:55:27.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef9a61640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef40b5530 con 0x7f0ef40cf2d0 2023-09-12T14:55:27.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef9a61640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef40b5810 con 0x7f0ef40cf6b0 2023-09-12T14:55:27.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef37fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ef40d1680 0x7f0ef415adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:27.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef37fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ef40d1680 0x7f0ef415adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56026/0 (socket says 172.21.15.131:56026) 2023-09-12T14:55:27.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef37fe640 1 -- 172.21.15.131:0/1800723997 learned_addr learned my addr 172.21.15.131:0/1800723997 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:27.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef2ffd640 1 --2- 172.21.15.131:0/1800723997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ef40cf2d0 0x7f0ef40b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:27.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef27fc640 1 --2- 172.21.15.131:0/1800723997 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ef40cf6b0 0x7f0ef4155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:27.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef37fe640 1 -- 172.21.15.131:0/1800723997 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ef40cf6b0 msgr2=0x7f0ef4155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:27.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef37fe640 1 --2- 172.21.15.131:0/1800723997 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ef40cf6b0 0x7f0ef4155290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:27.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef37fe640 1 -- 172.21.15.131:0/1800723997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ef40cf2d0 msgr2=0x7f0ef40b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:27.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef37fe640 1 --2- 172.21.15.131:0/1800723997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ef40cf2d0 0x7f0ef40b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:27.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef37fe640 1 -- 172.21.15.131:0/1800723997 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee4009560 con 0x7f0ef40d1680 2023-09-12T14:55:27.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef37fe640 1 --2- 172.21.15.131:0/1800723997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ef40d1680 0x7f0ef415adf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00c780 tx=0x7f0eec00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:27.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0edbfff640 1 -- 172.21.15.131:0/1800723997 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec004b70 con 0x7f0ef40d1680 2023-09-12T14:55:27.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0edbfff640 1 -- 172.21.15.131:0/1800723997 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0eec00ce90 con 0x7f0ef40d1680 2023-09-12T14:55:27.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef9a61640 1 -- 172.21.15.131:0/1800723997 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef40c80e0 con 0x7f0ef40d1680 2023-09-12T14:55:27.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0edbfff640 1 -- 172.21.15.131:0/1800723997 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec00aab0 con 0x7f0ef40d1680 2023-09-12T14:55:27.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.132+0000 7f0ef9a61640 1 -- 172.21.15.131:0/1800723997 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef40b5220 con 0x7f0ef40d1680 2023-09-12T14:55:27.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.136+0000 7f0ef9a61640 1 -- 172.21.15.131:0/1800723997 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ef4073580 con 0x7f0ef40d1680 2023-09-12T14:55:27.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.136+0000 7f0edbfff640 1 -- 172.21.15.131:0/1800723997 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f0eec00ac10 con 0x7f0ef40d1680 2023-09-12T14:55:27.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.136+0000 7f0edbfff640 1 --2- 172.21.15.131:0/1800723997 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0ee0065b20 0x7f0ee0067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:27.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.136+0000 7f0ef2ffd640 1 --2- 172.21.15.131:0/1800723997 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0ee0065b20 0x7f0ee0067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:27.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.136+0000 7f0edbfff640 1 -- 172.21.15.131:0/1800723997 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0eec0839c0 con 0x7f0ef40d1680 2023-09-12T14:55:27.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.136+0000 7f0ef2ffd640 1 --2- 172.21.15.131:0/1800723997 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0ee0065b20 0x7f0ee0067fe0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0ee40af000 tx=0x7f0ee401b0c0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:27.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.140+0000 7f0edbfff640 1 -- 172.21.15.131:0/1800723997 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f0eec051310 con 0x7f0ef40d1680 2023-09-12T14:55:27.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.292+0000 7f0ef9a61640 1 -- 172.21.15.131:0/1800723997 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ef40d0250 con 0x7f0ef40d1680 2023-09-12T14:55:27.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.296+0000 7f0edbfff640 1 -- 172.21.15.131:0/1800723997 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f0eec054930 con 0x7f0ef40d1680 2023-09-12T14:55:27.300 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:27.300 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:54:59.694778+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:54:55.689983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T14:54:57.029447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:55:27.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.296+0000 7f0ef9a61640 1 -- 172.21.15.131:0/1800723997 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0ee0065b20 msgr2=0x7f0ee0067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:27.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.296+0000 7f0ef9a61640 1 --2- 172.21.15.131:0/1800723997 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0ee0065b20 0x7f0ee0067fe0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0ee40af000 tx=0x7f0ee401b0c0 comp rx=0 tx=0).stop 2023-09-12T14:55:27.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.296+0000 7f0ef9a61640 1 -- 172.21.15.131:0/1800723997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ef40d1680 msgr2=0x7f0ef415adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:27.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.296+0000 7f0ef9a61640 1 --2- 172.21.15.131:0/1800723997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ef40d1680 0x7f0ef415adf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00c780 tx=0x7f0eec00cc40 comp rx=0 tx=0).stop 2023-09-12T14:55:27.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.296+0000 7f0ef9a61640 1 -- 172.21.15.131:0/1800723997 shutdown_connections 2023-09-12T14:55:27.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.296+0000 7f0ef9a61640 1 --2- 172.21.15.131:0/1800723997 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0ee0065b20 0x7f0ee0067fe0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:27.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.296+0000 7f0ef9a61640 1 --2- 172.21.15.131:0/1800723997 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ef40cf6b0 0x7f0ef4155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:27.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.296+0000 7f0ef9a61640 1 --2- 172.21.15.131:0/1800723997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ef40d1680 0x7f0ef415adf0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:27.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.296+0000 7f0ef9a61640 1 --2- 172.21.15.131:0/1800723997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ef40cf2d0 0x7f0ef40b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:27.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.296+0000 7f0ef9a61640 1 -- 172.21.15.131:0/1800723997 >> 172.21.15.131:0/1800723997 conn(0x7f0ef409a200 msgr2=0x7f0ef40cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:27.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.296+0000 7f0ef9a61640 1 -- 172.21.15.131:0/1800723997 shutdown_connections 2023-09-12T14:55:27.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.296+0000 7f0ef9a61640 1 -- 172.21.15.131:0/1800723997 wait complete. 2023-09-12T14:55:27.318 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T14:54:45.654004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T14:54:55.689983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T14:54:57.029447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-12T14:55:27.319 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T14:55:27.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.444+0000 7f9cb1079640 1 -- 172.21.15.131:0/987021362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cac156930 msgr2=0x7f9cac159dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:27.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.444+0000 7f9cb1079640 1 --2- 172.21.15.131:0/987021362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cac156930 0x7f9cac159dc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9cac06d170 tx=0x7f9c9c01c870 comp rx=0 tx=0).stop 2023-09-12T14:55:27.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.444+0000 7f9cb1079640 1 -- 172.21.15.131:0/987021362 shutdown_connections 2023-09-12T14:55:27.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.444+0000 7f9cb1079640 1 --2- 172.21.15.131:0/987021362 >> v2:172.21.15.187:3300/0 conn(0x7f9cac0c9370 0x7f9cac0c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:27.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.444+0000 7f9cb1079640 1 --2- 172.21.15.131:0/987021362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cac0abb40 0x7f9cac06b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:27.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.444+0000 7f9cb1079640 1 --2- 172.21.15.131:0/987021362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cac156930 0x7f9cac159dc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:27.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.444+0000 7f9cb1079640 1 -- 172.21.15.131:0/987021362 >> 172.21.15.131:0/987021362 conn(0x7f9cac069840 msgr2=0x7f9cac0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:27.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.444+0000 7f9cb1079640 1 -- 172.21.15.131:0/987021362 shutdown_connections 2023-09-12T14:55:27.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.444+0000 7f9cb1079640 1 -- 172.21.15.131:0/987021362 wait complete. 2023-09-12T14:55:27.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.448+0000 7f9cb1079640 1 Processor -- start 2023-09-12T14:55:27.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.448+0000 7f9cb1079640 1 -- start start 2023-09-12T14:55:27.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.448+0000 7f9cb1079640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cac0abb40 0x7f9cac144fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:27.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.448+0000 7f9cb1079640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cac0c9370 0x7f9cac15b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:27.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.448+0000 7f9cb1079640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cac156930 0x7f9cac146960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:27.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.448+0000 7f9cb1079640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac1f32a0 con 0x7f9cac0abb40 2023-09-12T14:55:27.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.448+0000 7f9cb1079640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac1f2720 con 0x7f9cac0c9370 2023-09-12T14:55:27.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.448+0000 7f9cb1079640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac1f2a00 con 0x7f9cac156930 2023-09-12T14:55:27.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.448+0000 7f9caad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cac0abb40 0x7f9cac144fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:27.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.448+0000 7f9caad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cac0abb40 0x7f9cac144fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56052/0 (socket says 172.21.15.131:56052) 2023-09-12T14:55:27.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.448+0000 7f9caad9d640 1 -- 172.21.15.131:0/3953077798 learned_addr learned my addr 172.21.15.131:0/3953077798 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:27.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.448+0000 7f9caad9d640 1 -- 172.21.15.131:0/3953077798 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cac156930 msgr2=0x7f9cac146960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:27.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.448+0000 7f9caad9d640 1 --2- 172.21.15.131:0/3953077798 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cac156930 0x7f9cac146960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:27.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.448+0000 7f9caad9d640 1 -- 172.21.15.131:0/3953077798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cac0c9370 msgr2=0x7f9cac15b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:27.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.448+0000 7f9caa59c640 1 --2- 172.21.15.131:0/3953077798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cac0c9370 0x7f9cac15b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:27.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.448+0000 7f9caad9d640 1 --2- 172.21.15.131:0/3953077798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cac0c9370 0x7f9cac15b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:27.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.448+0000 7f9caad9d640 1 -- 172.21.15.131:0/3953077798 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9c009560 con 0x7f9cac0abb40 2023-09-12T14:55:27.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.448+0000 7f9caad9d640 1 --2- 172.21.15.131:0/3953077798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cac0abb40 0x7f9cac144fd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c000c10 tx=0x7f9c9c0a5790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:27.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.448+0000 7f9ca37fe640 1 -- 172.21.15.131:0/3953077798 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c005220 con 0x7f9cac0abb40 2023-09-12T14:55:27.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.448+0000 7f9ca37fe640 1 -- 172.21.15.131:0/3953077798 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9c9c005490 con 0x7f9cac0abb40 2023-09-12T14:55:27.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.448+0000 7f9cb1079640 1 -- 172.21.15.131:0/3953077798 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cac1f2ce0 con 0x7f9cac0abb40 2023-09-12T14:55:27.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.448+0000 7f9ca37fe640 1 -- 172.21.15.131:0/3953077798 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c004930 con 0x7f9cac0abb40 2023-09-12T14:55:27.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.448+0000 7f9cb1079640 1 -- 172.21.15.131:0/3953077798 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cac145510 con 0x7f9cac0abb40 2023-09-12T14:55:27.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.448+0000 7f9cb1079640 1 -- 172.21.15.131:0/3953077798 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cac1f59c0 con 0x7f9cac0abb40 2023-09-12T14:55:27.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.452+0000 7f9ca37fe640 1 -- 172.21.15.131:0/3953077798 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f9c9c0b4070 con 0x7f9cac0abb40 2023-09-12T14:55:27.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.452+0000 7f9ca37fe640 1 --2- 172.21.15.131:0/3953077798 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9c8c065b20 0x7f9c8c067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:27.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.452+0000 7f9ca37fe640 1 -- 172.21.15.131:0/3953077798 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c9c11e070 con 0x7f9cac0abb40 2023-09-12T14:55:27.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.452+0000 7f9caa59c640 1 --2- 172.21.15.131:0/3953077798 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9c8c065b20 0x7f9c8c067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:27.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.452+0000 7f9caa59c640 1 --2- 172.21.15.131:0/3953077798 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9c8c065b20 0x7f9c8c067fe0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9cac0c8c40 tx=0x7f9c94008040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:27.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.456+0000 7f9ca37fe640 1 -- 172.21.15.131:0/3953077798 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f9c9c0ebb30 con 0x7f9cac0abb40 2023-09-12T14:55:27.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.596+0000 7f9cb1079640 1 -- 172.21.15.131:0/3953077798 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9cac073580 con 0x7f9cac0abb40 2023-09-12T14:55:27.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.596+0000 7f9ca37fe640 1 -- 172.21.15.131:0/3953077798 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9c9c0bc020 con 0x7f9cac0abb40 2023-09-12T14:55:27.599 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-09-12T14:55:27.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.596+0000 7f9cb1079640 1 -- 172.21.15.131:0/3953077798 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9c8c065b20 msgr2=0x7f9c8c067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:27.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.596+0000 7f9cb1079640 1 --2- 172.21.15.131:0/3953077798 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9c8c065b20 0x7f9c8c067fe0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9cac0c8c40 tx=0x7f9c94008040 comp rx=0 tx=0).stop 2023-09-12T14:55:27.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.596+0000 7f9cb1079640 1 -- 172.21.15.131:0/3953077798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cac0abb40 msgr2=0x7f9cac144fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:27.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.596+0000 7f9cb1079640 1 --2- 172.21.15.131:0/3953077798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cac0abb40 0x7f9cac144fd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c000c10 tx=0x7f9c9c0a5790 comp rx=0 tx=0).stop 2023-09-12T14:55:27.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.596+0000 7f9cb1079640 1 -- 172.21.15.131:0/3953077798 shutdown_connections 2023-09-12T14:55:27.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.596+0000 7f9cb1079640 1 --2- 172.21.15.131:0/3953077798 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9c8c065b20 0x7f9c8c067fe0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:27.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.596+0000 7f9cb1079640 1 --2- 172.21.15.131:0/3953077798 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cac156930 0x7f9cac146960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:27.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.596+0000 7f9cb1079640 1 --2- 172.21.15.131:0/3953077798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cac0c9370 0x7f9cac15b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:27.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.596+0000 7f9cb1079640 1 --2- 172.21.15.131:0/3953077798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cac0abb40 0x7f9cac144fd0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:27.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.596+0000 7f9cb1079640 1 -- 172.21.15.131:0/3953077798 >> 172.21.15.131:0/3953077798 conn(0x7f9cac069840 msgr2=0x7f9cac14bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:27.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.596+0000 7f9cb1079640 1 -- 172.21.15.131:0/3953077798 shutdown_connections 2023-09-12T14:55:27.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.600+0000 7f9cb1079640 1 -- 172.21.15.131:0/3953077798 wait complete. 2023-09-12T14:55:27.616 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T14:55:27.751 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61cc6f2640 1 -- 172.21.15.131:0/2712492201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61b0003980 msgr2=0x7f61b0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:27.751 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61cc6f2640 1 --2- 172.21.15.131:0/2712492201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61b0003980 0x7f61b0003da0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f61c0009b90 tx=0x7f61c001d250 comp rx=0 tx=0).stop 2023-09-12T14:55:27.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61cac8f640 1 -- 172.21.15.131:0/2712492201 reap_dead start 2023-09-12T14:55:27.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61cc6f2640 1 -- 172.21.15.131:0/2712492201 shutdown_connections 2023-09-12T14:55:27.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61cc6f2640 1 -- 172.21.15.131:0/2712492201 >> 172.21.15.131:0/2712492201 conn(0x7f61c4069840 msgr2=0x7f61c40cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:27.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61cc6f2640 1 -- 172.21.15.131:0/2712492201 shutdown_connections 2023-09-12T14:55:27.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61cc6f2640 1 -- 172.21.15.131:0/2712492201 wait complete. 2023-09-12T14:55:27.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61cc6f2640 1 Processor -- start 2023-09-12T14:55:27.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61cc6f2640 1 -- start start 2023-09-12T14:55:27.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61cc6f2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61b0003980 0x7f61c41453f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:27.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61cc6f2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c415fda0 0x7f61c4160220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:27.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61cc6f2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61c4152e90 0x7f61c4153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:27.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61cc6f2640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c4145930 con 0x7f61c415fda0 2023-09-12T14:55:27.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61cc6f2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c4153850 con 0x7f61b0003980 2023-09-12T14:55:27.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61cc6f2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c4160840 con 0x7f61c4152e90 2023-09-12T14:55:27.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61c9c8d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c415fda0 0x7f61c4160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:27.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61c9c8d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c415fda0 0x7f61c4160220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56082/0 (socket says 172.21.15.131:56082) 2023-09-12T14:55:27.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61c9c8d640 1 -- 172.21.15.131:0/378947726 learned_addr learned my addr 172.21.15.131:0/378947726 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:27.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61ca48e640 1 --2- 172.21.15.131:0/378947726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61b0003980 0x7f61c41453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:27.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61cac8f640 1 --2- 172.21.15.131:0/378947726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61c4152e90 0x7f61c4153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:27.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61c9c8d640 1 -- 172.21.15.131:0/378947726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61c4152e90 msgr2=0x7f61c4153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:27.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61c9c8d640 1 --2- 172.21.15.131:0/378947726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61c4152e90 0x7f61c4153310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:27.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61c9c8d640 1 -- 172.21.15.131:0/378947726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61b0003980 msgr2=0x7f61c41453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:27.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61c9c8d640 1 --2- 172.21.15.131:0/378947726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61b0003980 0x7f61c41453f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:27.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61c9c8d640 1 -- 172.21.15.131:0/378947726 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c4160b20 con 0x7f61c415fda0 2023-09-12T14:55:27.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61cac8f640 1 --2- 172.21.15.131:0/378947726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61c4152e90 0x7f61c4153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:27.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61c9c8d640 1 --2- 172.21.15.131:0/378947726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c415fda0 0x7f61c4160220 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f61b400e590 tx=0x7f61b400ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:27.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61baffd640 1 -- 172.21.15.131:0/378947726 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4010890 con 0x7f61c415fda0 2023-09-12T14:55:27.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61baffd640 1 -- 172.21.15.131:0/378947726 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f61b4010b00 con 0x7f61c415fda0 2023-09-12T14:55:27.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61cc6f2640 1 -- 172.21.15.131:0/378947726 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c4160e00 con 0x7f61c415fda0 2023-09-12T14:55:27.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61baffd640 1 -- 172.21.15.131:0/378947726 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4011460 con 0x7f61c415fda0 2023-09-12T14:55:27.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.744+0000 7f61cc6f2640 1 -- 172.21.15.131:0/378947726 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c41613c0 con 0x7f61c415fda0 2023-09-12T14:55:27.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.748+0000 7f61cc6f2640 1 -- 172.21.15.131:0/378947726 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c4073580 con 0x7f61c415fda0 2023-09-12T14:55:27.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.748+0000 7f61baffd640 1 -- 172.21.15.131:0/378947726 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f61b40115c0 con 0x7f61c415fda0 2023-09-12T14:55:27.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.748+0000 7f61baffd640 1 --2- 172.21.15.131:0/378947726 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f61a8065b20 0x7f61a8067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:27.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.748+0000 7f61ca48e640 1 --2- 172.21.15.131:0/378947726 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f61a8065b20 0x7f61a8067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:27.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.748+0000 7f61baffd640 1 -- 172.21.15.131:0/378947726 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61b4083a80 con 0x7f61c415fda0 2023-09-12T14:55:27.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.748+0000 7f61ca48e640 1 --2- 172.21.15.131:0/378947726 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f61a8065b20 0x7f61a8067fe0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f61c00ae000 tx=0x7f61c0019ff0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:27.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.752+0000 7f61baffd640 1 -- 172.21.15.131:0/378947726 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f61b40513d0 con 0x7f61c415fda0 2023-09-12T14:55:27.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.896+0000 7f61cc6f2640 1 -- 172.21.15.131:0/378947726 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f61c40cd4d0 con 0x7f61c415fda0 2023-09-12T14:55:27.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.896+0000 7f61baffd640 1 -- 172.21.15.131:0/378947726 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f61b401f020 con 0x7f61c415fda0 2023-09-12T14:55:27.900 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-09-12T14:55:27.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.896+0000 7f61cc6f2640 1 -- 172.21.15.131:0/378947726 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f61a8065b20 msgr2=0x7f61a8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:27.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.896+0000 7f61cc6f2640 1 --2- 172.21.15.131:0/378947726 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f61a8065b20 0x7f61a8067fe0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f61c00ae000 tx=0x7f61c0019ff0 comp rx=0 tx=0).stop 2023-09-12T14:55:27.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.896+0000 7f61cc6f2640 1 -- 172.21.15.131:0/378947726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c415fda0 msgr2=0x7f61c4160220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:27.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.896+0000 7f61cc6f2640 1 --2- 172.21.15.131:0/378947726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c415fda0 0x7f61c4160220 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f61b400e590 tx=0x7f61b400ea50 comp rx=0 tx=0).stop 2023-09-12T14:55:27.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.896+0000 7f61cc6f2640 1 -- 172.21.15.131:0/378947726 shutdown_connections 2023-09-12T14:55:27.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.896+0000 7f61cc6f2640 1 --2- 172.21.15.131:0/378947726 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f61a8065b20 0x7f61a8067fe0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:27.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.896+0000 7f61cc6f2640 1 --2- 172.21.15.131:0/378947726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61c4152e90 0x7f61c4153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:27.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.896+0000 7f61cc6f2640 1 --2- 172.21.15.131:0/378947726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c415fda0 0x7f61c4160220 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:27.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.896+0000 7f61cc6f2640 1 --2- 172.21.15.131:0/378947726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61b0003980 0x7f61c41453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:27.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.896+0000 7f61cc6f2640 1 -- 172.21.15.131:0/378947726 >> 172.21.15.131:0/378947726 conn(0x7f61c4069840 msgr2=0x7f61c41540c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:27.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.896+0000 7f61cc6f2640 1 -- 172.21.15.131:0/378947726 shutdown_connections 2023-09-12T14:55:27.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:27.896+0000 7f61cc6f2640 1 -- 172.21.15.131:0/378947726 wait complete. 2023-09-12T14:55:27.916 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T14:55:28.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.040+0000 7f26ca926640 1 -- 172.21.15.131:0/2270486416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26c406c060 msgr2=0x7f26c41626d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:28.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.040+0000 7f26ca926640 1 --2- 172.21.15.131:0/2270486416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26c406c060 0x7f26c41626d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f26bc009bf0 tx=0x7f26bc01cb90 comp rx=0 tx=0).stop 2023-09-12T14:55:28.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.040+0000 7f26ca926640 1 -- 172.21.15.131:0/2270486416 shutdown_connections 2023-09-12T14:55:28.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.040+0000 7f26ca926640 1 --2- 172.21.15.131:0/2270486416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26c406c060 0x7f26c41626d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.040+0000 7f26ca926640 1 -- 172.21.15.131:0/2270486416 >> 172.21.15.131:0/2270486416 conn(0x7f26c4069840 msgr2=0x7f26c40cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:28.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.040+0000 7f26ca926640 1 -- 172.21.15.131:0/2270486416 shutdown_connections 2023-09-12T14:55:28.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.040+0000 7f26ca926640 1 -- 172.21.15.131:0/2270486416 wait complete. 2023-09-12T14:55:28.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.044+0000 7f26ca926640 1 Processor -- start 2023-09-12T14:55:28.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.044+0000 7f26ca926640 1 -- start start 2023-09-12T14:55:28.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.044+0000 7f26ca926640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26c406c060 0x7f26c415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:28.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.044+0000 7f26ca926640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26c40ab730 0x7f26c4159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:28.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.044+0000 7f26ca926640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26c4158180 0x7f26c413d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:28.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.044+0000 7f26ca926640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c415a680 con 0x7f26c406c060 2023-09-12T14:55:28.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.044+0000 7f26ca926640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c4159780 con 0x7f26c4158180 2023-09-12T14:55:28.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.044+0000 7f26ca926640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26c413d920 con 0x7f26c40ab730 2023-09-12T14:55:28.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.044+0000 7f26c3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26c406c060 0x7f26c415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:28.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.044+0000 7f26c3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26c406c060 0x7f26c415aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56088/0 (socket says 172.21.15.131:56088) 2023-09-12T14:55:28.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.044+0000 7f26c3fff640 1 -- 172.21.15.131:0/3287922553 learned_addr learned my addr 172.21.15.131:0/3287922553 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:28.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.044+0000 7f26c8ec3640 1 --2- 172.21.15.131:0/3287922553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26c4158180 0x7f26c413d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:28.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.044+0000 7f26c37fe640 1 --2- 172.21.15.131:0/3287922553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26c40ab730 0x7f26c4159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:28.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.044+0000 7f26c3fff640 1 -- 172.21.15.131:0/3287922553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26c40ab730 msgr2=0x7f26c4159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:28.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.044+0000 7f26c3fff640 1 --2- 172.21.15.131:0/3287922553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26c40ab730 0x7f26c4159240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.044+0000 7f26c3fff640 1 -- 172.21.15.131:0/3287922553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26c4158180 msgr2=0x7f26c413d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:28.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.044+0000 7f26c3fff640 1 --2- 172.21.15.131:0/3287922553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26c4158180 0x7f26c413d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.044+0000 7f26c3fff640 1 -- 172.21.15.131:0/3287922553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26bc009820 con 0x7f26c406c060 2023-09-12T14:55:28.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.044+0000 7f26c3fff640 1 --2- 172.21.15.131:0/3287922553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26c406c060 0x7f26c415aaa0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f26b800c4b0 tx=0x7f26b800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:28.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.044+0000 7f26c0ff9640 1 -- 172.21.15.131:0/3287922553 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b8012830 con 0x7f26c406c060 2023-09-12T14:55:28.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.044+0000 7f26c0ff9640 1 -- 172.21.15.131:0/3287922553 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f26b8012a80 con 0x7f26c406c060 2023-09-12T14:55:28.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.044+0000 7f26ca926640 1 -- 172.21.15.131:0/3287922553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26c4159a00 con 0x7f26c406c060 2023-09-12T14:55:28.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.044+0000 7f26c0ff9640 1 -- 172.21.15.131:0/3287922553 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b8012e70 con 0x7f26c406c060 2023-09-12T14:55:28.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.044+0000 7f26ca926640 1 -- 172.21.15.131:0/3287922553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c4159fc0 con 0x7f26c406c060 2023-09-12T14:55:28.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.044+0000 7f26ca926640 1 -- 172.21.15.131:0/3287922553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26c4073580 con 0x7f26c406c060 2023-09-12T14:55:28.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.044+0000 7f26c0ff9640 1 -- 172.21.15.131:0/3287922553 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f26b801a3b0 con 0x7f26c406c060 2023-09-12T14:55:28.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.048+0000 7f26c0ff9640 1 --2- 172.21.15.131:0/3287922553 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f26b0065b20 0x7f26b0067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:28.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.048+0000 7f26c0ff9640 1 -- 172.21.15.131:0/3287922553 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26b80833d0 con 0x7f26c406c060 2023-09-12T14:55:28.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.048+0000 7f26c37fe640 1 --2- 172.21.15.131:0/3287922553 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f26b0065b20 0x7f26b0067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:28.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.048+0000 7f26c37fe640 1 --2- 172.21.15.131:0/3287922553 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f26b0065b20 0x7f26b0067fe0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f26c414da20 tx=0x7f26b4009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:28.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.052+0000 7f26c0ff9640 1 -- 172.21.15.131:0/3287922553 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f26b8050d30 con 0x7f26c406c060 2023-09-12T14:55:28.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.192+0000 7f26ca926640 1 -- 172.21.15.131:0/3287922553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f26c40cb360 con 0x7f26c406c060 2023-09-12T14:55:28.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.192+0000 7f26c0ff9640 1 -- 172.21.15.131:0/3287922553 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f26b8054350 con 0x7f26c406c060 2023-09-12T14:55:28.197 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-09-12T14:55:28.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.196+0000 7f26ca926640 1 -- 172.21.15.131:0/3287922553 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f26b0065b20 msgr2=0x7f26b0067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:28.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.196+0000 7f26ca926640 1 --2- 172.21.15.131:0/3287922553 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f26b0065b20 0x7f26b0067fe0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f26c414da20 tx=0x7f26b4009040 comp rx=0 tx=0).stop 2023-09-12T14:55:28.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.196+0000 7f26ca926640 1 -- 172.21.15.131:0/3287922553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26c406c060 msgr2=0x7f26c415aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:28.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.196+0000 7f26ca926640 1 --2- 172.21.15.131:0/3287922553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26c406c060 0x7f26c415aaa0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f26b800c4b0 tx=0x7f26b800c970 comp rx=0 tx=0).stop 2023-09-12T14:55:28.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.196+0000 7f26ca926640 1 -- 172.21.15.131:0/3287922553 shutdown_connections 2023-09-12T14:55:28.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.196+0000 7f26ca926640 1 --2- 172.21.15.131:0/3287922553 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f26b0065b20 0x7f26b0067fe0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.196+0000 7f26ca926640 1 --2- 172.21.15.131:0/3287922553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26c40ab730 0x7f26c4159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.196+0000 7f26ca926640 1 --2- 172.21.15.131:0/3287922553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26c4158180 0x7f26c413d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.196+0000 7f26ca926640 1 --2- 172.21.15.131:0/3287922553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26c406c060 0x7f26c415aaa0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.196+0000 7f26ca926640 1 -- 172.21.15.131:0/3287922553 >> 172.21.15.131:0/3287922553 conn(0x7f26c4069840 msgr2=0x7f26c406f870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:28.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.196+0000 7f26ca926640 1 -- 172.21.15.131:0/3287922553 shutdown_connections 2023-09-12T14:55:28.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.196+0000 7f26ca926640 1 -- 172.21.15.131:0/3287922553 wait complete. 2023-09-12T14:55:28.220 INFO:tasks.cephfs_test_runner:Starting test: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) test: 2023-09-12T14:55:28.221 INFO:tasks.cephfs_test_runner:test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-09-12T14:55:28.222 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:55:28.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.336+0000 7f143a8e6640 1 -- 172.21.15.131:0/643526392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14340afa40 msgr2=0x7f14340afeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:28.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.336+0000 7f143a8e6640 1 --2- 172.21.15.131:0/643526392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14340afa40 0x7f14340afeb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f142c009900 tx=0x7f142c01c830 comp rx=0 tx=0).stop 2023-09-12T14:55:28.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.336+0000 7f143a8e6640 1 -- 172.21.15.131:0/643526392 shutdown_connections 2023-09-12T14:55:28.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.336+0000 7f143a8e6640 1 --2- 172.21.15.131:0/643526392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14340afa40 0x7f14340afeb0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.336+0000 7f143a8e6640 1 --2- 172.21.15.131:0/643526392 >> v2:172.21.15.187:3300/0 conn(0x7f143406db90 0x7f1434070a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.336+0000 7f143a8e6640 1 -- 172.21.15.131:0/643526392 >> 172.21.15.131:0/643526392 conn(0x7f14340cdd80 msgr2=0x7f14340ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:28.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.336+0000 7f143a8e6640 1 -- 172.21.15.131:0/643526392 shutdown_connections 2023-09-12T14:55:28.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.336+0000 7f143a8e6640 1 -- 172.21.15.131:0/643526392 wait complete. 2023-09-12T14:55:28.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.336+0000 7f143a8e6640 1 Processor -- start 2023-09-12T14:55:28.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.336+0000 7f143a8e6640 1 -- start start 2023-09-12T14:55:28.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f143a8e6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f143406db90 0x7f14341f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:28.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f143a8e6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14340afa40 0x7f14341f7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:28.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f143a8e6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1434157bc0 0x7f143414d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:28.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f143a8e6640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14341f7ea0 con 0x7f1434157bc0 2023-09-12T14:55:28.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f143a8e6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1434159cb0 con 0x7f14340afa40 2023-09-12T14:55:28.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f143a8e6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14341599d0 con 0x7f143406db90 2023-09-12T14:55:28.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f1438e83640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1434157bc0 0x7f143414d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:28.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f1438e83640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1434157bc0 0x7f143414d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59716/0 (socket says 172.21.15.131:59716) 2023-09-12T14:55:28.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f1438e83640 1 -- 172.21.15.131:0/2651144417 learned_addr learned my addr 172.21.15.131:0/2651144417 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:28.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f1433fff640 1 --2- 172.21.15.131:0/2651144417 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f143406db90 0x7f14341f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:28.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f14337fe640 1 --2- 172.21.15.131:0/2651144417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14340afa40 0x7f14341f7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:28.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f1438e83640 1 -- 172.21.15.131:0/2651144417 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f143406db90 msgr2=0x7f14341f5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:28.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f1438e83640 1 --2- 172.21.15.131:0/2651144417 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f143406db90 0x7f14341f5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f1438e83640 1 -- 172.21.15.131:0/2651144417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14340afa40 msgr2=0x7f14341f7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:28.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f1438e83640 1 --2- 172.21.15.131:0/2651144417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14340afa40 0x7f14341f7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f1438e83640 1 -- 172.21.15.131:0/2651144417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142c009560 con 0x7f1434157bc0 2023-09-12T14:55:28.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f1438e83640 1 --2- 172.21.15.131:0/2651144417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1434157bc0 0x7f143414d170 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f142c0b0040 tx=0x7f142c0a66e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:28.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f1430ff9640 1 -- 172.21.15.131:0/2651144417 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c004240 con 0x7f1434157bc0 2023-09-12T14:55:28.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f1430ff9640 1 -- 172.21.15.131:0/2651144417 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f142c0043a0 con 0x7f1434157bc0 2023-09-12T14:55:28.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f143a8e6640 1 -- 172.21.15.131:0/2651144417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14340b1fd0 con 0x7f1434157bc0 2023-09-12T14:55:28.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f1430ff9640 1 -- 172.21.15.131:0/2651144417 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c004530 con 0x7f1434157bc0 2023-09-12T14:55:28.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f143a8e6640 1 -- 172.21.15.131:0/2651144417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1434161330 con 0x7f1434157bc0 2023-09-12T14:55:28.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f143a8e6640 1 -- 172.21.15.131:0/2651144417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14340cc500 con 0x7f1434157bc0 2023-09-12T14:55:28.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f1430ff9640 1 -- 172.21.15.131:0/2651144417 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f142c0052d0 con 0x7f1434157bc0 2023-09-12T14:55:28.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f1430ff9640 1 --2- 172.21.15.131:0/2651144417 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1414065b20 0x7f1414067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:28.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f1433fff640 1 --2- 172.21.15.131:0/2651144417 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1414065b20 0x7f1414067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:28.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f1430ff9640 1 -- 172.21.15.131:0/2651144417 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f142c0aa070 con 0x7f1434157bc0 2023-09-12T14:55:28.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.340+0000 7f1433fff640 1 --2- 172.21.15.131:0/2651144417 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1414065b20 0x7f1414067fe0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f14240015b0 tx=0x7f1424009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:28.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.344+0000 7f1430ff9640 1 -- 172.21.15.131:0/2651144417 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f142c0ebd70 con 0x7f1434157bc0 2023-09-12T14:55:28.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.496+0000 7f143a8e6640 1 -- 172.21.15.131:0/2651144417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14340b1e20 con 0x7f1434157bc0 2023-09-12T14:55:28.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.500+0000 7f1430ff9640 1 -- 172.21.15.131:0/2651144417 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f142c0ef390 con 0x7f1434157bc0 2023-09-12T14:55:28.505 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:28.505 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:54:59.694778+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:54:55.689983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T14:54:57.029447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:55:28.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.504+0000 7f143a8e6640 1 -- 172.21.15.131:0/2651144417 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1414065b20 msgr2=0x7f1414067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:28.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.504+0000 7f143a8e6640 1 --2- 172.21.15.131:0/2651144417 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1414065b20 0x7f1414067fe0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f14240015b0 tx=0x7f1424009040 comp rx=0 tx=0).stop 2023-09-12T14:55:28.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.504+0000 7f143a8e6640 1 -- 172.21.15.131:0/2651144417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1434157bc0 msgr2=0x7f143414d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:28.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.504+0000 7f143a8e6640 1 --2- 172.21.15.131:0/2651144417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1434157bc0 0x7f143414d170 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f142c0b0040 tx=0x7f142c0a66e0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.504+0000 7f143a8e6640 1 -- 172.21.15.131:0/2651144417 shutdown_connections 2023-09-12T14:55:28.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.504+0000 7f143a8e6640 1 --2- 172.21.15.131:0/2651144417 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1414065b20 0x7f1414067fe0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.504+0000 7f143a8e6640 1 --2- 172.21.15.131:0/2651144417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1434157bc0 0x7f143414d170 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.504+0000 7f143a8e6640 1 --2- 172.21.15.131:0/2651144417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14340afa40 0x7f14341f7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.504+0000 7f143a8e6640 1 --2- 172.21.15.131:0/2651144417 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f143406db90 0x7f14341f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.504+0000 7f143a8e6640 1 -- 172.21.15.131:0/2651144417 >> 172.21.15.131:0/2651144417 conn(0x7f14340cdd80 msgr2=0x7f14340b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:28.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.504+0000 7f143a8e6640 1 -- 172.21.15.131:0/2651144417 shutdown_connections 2023-09-12T14:55:28.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.504+0000 7f143a8e6640 1 -- 172.21.15.131:0/2651144417 wait complete. 2023-09-12T14:55:28.523 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T14:54:45.654004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T14:54:55.689983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T14:54:57.029447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-12T14:55:28.524 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T14:55:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f553cd8f640 1 -- 172.21.15.131:0/562694573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55380ab730 msgr2=0x7f55380abb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f553cd8f640 1 --2- 172.21.15.131:0/562694573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55380ab730 0x7f55380abb10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f55380cd8f0 tx=0x7f552801c7b0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f553cd8f640 1 -- 172.21.15.131:0/562694573 shutdown_connections 2023-09-12T14:55:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f553cd8f640 1 --2- 172.21.15.131:0/562694573 >> v2:172.21.15.187:3300/0 conn(0x7f553806b050 0x7f553806b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f553cd8f640 1 --2- 172.21.15.131:0/562694573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55380ab730 0x7f55380abb10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f553cd8f640 1 -- 172.21.15.131:0/562694573 >> 172.21.15.131:0/562694573 conn(0x7f5538069840 msgr2=0x7f55380cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f553cd8f640 1 -- 172.21.15.131:0/562694573 shutdown_connections 2023-09-12T14:55:28.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f553cd8f640 1 -- 172.21.15.131:0/562694573 wait complete. 2023-09-12T14:55:28.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f553cd8f640 1 Processor -- start 2023-09-12T14:55:28.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f553cd8f640 1 -- start start 2023-09-12T14:55:28.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f553cd8f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f553806b050 0x7f553814e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:28.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f553cd8f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55380ab730 0x7f5538156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:28.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f553cd8f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5538158180 0x7f553814e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:28.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f553cd8f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5538153800 con 0x7f553806b050 2023-09-12T14:55:28.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f553cd8f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5538152a40 con 0x7f55380ab730 2023-09-12T14:55:28.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f553cd8f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f553814e550 con 0x7f5538158180 2023-09-12T14:55:28.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f553659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f553806b050 0x7f553814e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:28.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f553659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f553806b050 0x7f553814e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59726/0 (socket says 172.21.15.131:59726) 2023-09-12T14:55:28.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f553659c640 1 -- 172.21.15.131:0/1278675571 learned_addr learned my addr 172.21.15.131:0/1278675571 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:28.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f5536d9d640 1 --2- 172.21.15.131:0/1278675571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5538158180 0x7f553814e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:28.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f5535d9b640 1 --2- 172.21.15.131:0/1278675571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55380ab730 0x7f5538156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:28.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f553659c640 1 -- 172.21.15.131:0/1278675571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5538158180 msgr2=0x7f553814e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:28.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f553659c640 1 --2- 172.21.15.131:0/1278675571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5538158180 0x7f553814e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f553659c640 1 -- 172.21.15.131:0/1278675571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55380ab730 msgr2=0x7f5538156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:28.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f553659c640 1 --2- 172.21.15.131:0/1278675571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55380ab730 0x7f5538156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f553659c640 1 -- 172.21.15.131:0/1278675571 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55280094a0 con 0x7f553806b050 2023-09-12T14:55:28.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f5536d9d640 1 --2- 172.21.15.131:0/1278675571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5538158180 0x7f553814e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:28.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f5535d9b640 1 --2- 172.21.15.131:0/1278675571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55380ab730 0x7f5538156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:28.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f553659c640 1 --2- 172.21.15.131:0/1278675571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f553806b050 0x7f553814e730 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f552801cc90 tx=0x7f5528004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:28.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f5526ffd640 1 -- 172.21.15.131:0/1278675571 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528003800 con 0x7f553806b050 2023-09-12T14:55:28.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f5526ffd640 1 -- 172.21.15.131:0/1278675571 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5528003a50 con 0x7f553806b050 2023-09-12T14:55:28.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f553cd8f640 1 -- 172.21.15.131:0/1278675571 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55381574b0 con 0x7f553806b050 2023-09-12T14:55:28.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f5526ffd640 1 -- 172.21.15.131:0/1278675571 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528003e00 con 0x7f553806b050 2023-09-12T14:55:28.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.652+0000 7f553cd8f640 1 -- 172.21.15.131:0/1278675571 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5538153140 con 0x7f553806b050 2023-09-12T14:55:28.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.656+0000 7f553cd8f640 1 -- 172.21.15.131:0/1278675571 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5538073580 con 0x7f553806b050 2023-09-12T14:55:28.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.656+0000 7f5526ffd640 1 -- 172.21.15.131:0/1278675571 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f55280c2460 con 0x7f553806b050 2023-09-12T14:55:28.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.656+0000 7f5526ffd640 1 --2- 172.21.15.131:0/1278675571 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5518065b20 0x7f5518067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:28.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.656+0000 7f5526ffd640 1 -- 172.21.15.131:0/1278675571 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f552811e5b0 con 0x7f553806b050 2023-09-12T14:55:28.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.656+0000 7f5535d9b640 1 --2- 172.21.15.131:0/1278675571 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5518065b20 0x7f5518067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:28.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.656+0000 7f5535d9b640 1 --2- 172.21.15.131:0/1278675571 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5518065b20 0x7f5518067fe0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5538150d40 tx=0x7f552c009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:28.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.660+0000 7f5526ffd640 1 -- 172.21.15.131:0/1278675571 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f55280ec020 con 0x7f553806b050 2023-09-12T14:55:28.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.812+0000 7f553cd8f640 1 -- 172.21.15.131:0/1278675571 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f55380b2110 con 0x7f553806b050 2023-09-12T14:55:28.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.816+0000 7f5526ffd640 1 -- 172.21.15.131:0/1278675571 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f55280ef640 con 0x7f553806b050 2023-09-12T14:55:28.818 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-09-12T14:55:28.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.816+0000 7f553cd8f640 1 -- 172.21.15.131:0/1278675571 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5518065b20 msgr2=0x7f5518067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:28.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.816+0000 7f553cd8f640 1 --2- 172.21.15.131:0/1278675571 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5518065b20 0x7f5518067fe0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5538150d40 tx=0x7f552c009040 comp rx=0 tx=0).stop 2023-09-12T14:55:28.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.816+0000 7f553cd8f640 1 -- 172.21.15.131:0/1278675571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f553806b050 msgr2=0x7f553814e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:28.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.816+0000 7f553cd8f640 1 --2- 172.21.15.131:0/1278675571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f553806b050 0x7f553814e730 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f552801cc90 tx=0x7f5528004600 comp rx=0 tx=0).stop 2023-09-12T14:55:28.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.816+0000 7f553cd8f640 1 -- 172.21.15.131:0/1278675571 shutdown_connections 2023-09-12T14:55:28.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.816+0000 7f553cd8f640 1 --2- 172.21.15.131:0/1278675571 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5518065b20 0x7f5518067fe0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.816+0000 7f553cd8f640 1 --2- 172.21.15.131:0/1278675571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5538158180 0x7f553814e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.816+0000 7f553cd8f640 1 --2- 172.21.15.131:0/1278675571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55380ab730 0x7f5538156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.816+0000 7f553cd8f640 1 --2- 172.21.15.131:0/1278675571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f553806b050 0x7f553814e730 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.816+0000 7f553cd8f640 1 -- 172.21.15.131:0/1278675571 >> 172.21.15.131:0/1278675571 conn(0x7f5538069840 msgr2=0x7f55380cac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:28.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.816+0000 7f553cd8f640 1 -- 172.21.15.131:0/1278675571 shutdown_connections 2023-09-12T14:55:28.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.816+0000 7f553cd8f640 1 -- 172.21.15.131:0/1278675571 wait complete. 2023-09-12T14:55:28.837 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T14:55:28.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36eb1b7640 1 -- 172.21.15.131:0/950243408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36e40b09c0 msgr2=0x7f36e4161ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:28.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36eb1b7640 1 --2- 172.21.15.131:0/950243408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36e40b09c0 0x7f36e4161ee0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f36dc009bc0 tx=0x7f36dc01cb10 comp rx=0 tx=0).stop 2023-09-12T14:55:28.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36eb1b7640 1 -- 172.21.15.131:0/950243408 shutdown_connections 2023-09-12T14:55:28.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36eb1b7640 1 --2- 172.21.15.131:0/950243408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36e40b09c0 0x7f36e4161ee0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36eb1b7640 1 -- 172.21.15.131:0/950243408 >> 172.21.15.131:0/950243408 conn(0x7f36e4069840 msgr2=0x7f36e40cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:28.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36eb1b7640 1 -- 172.21.15.131:0/950243408 shutdown_connections 2023-09-12T14:55:28.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36eb1b7640 1 -- 172.21.15.131:0/950243408 wait complete. 2023-09-12T14:55:28.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36eb1b7640 1 Processor -- start 2023-09-12T14:55:28.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36eb1b7640 1 -- start start 2023-09-12T14:55:28.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36eb1b7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36e40b09c0 0x7f36e4144a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:28.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36eb1b7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36e40b2600 0x7f36e4153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:28.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36eb1b7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36e41574d0 0x7f36e41542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:28.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36eb1b7640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e41f7e50 con 0x7f36e40b2600 2023-09-12T14:55:28.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36eb1b7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e41f4fa0 con 0x7f36e41574d0 2023-09-12T14:55:28.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36eb1b7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e4153590 con 0x7f36e40b09c0 2023-09-12T14:55:28.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36dbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36e40b2600 0x7f36e4153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:28.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36dbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36e40b2600 0x7f36e4153d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59738/0 (socket says 172.21.15.131:59738) 2023-09-12T14:55:28.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36dbfff640 1 -- 172.21.15.131:0/2477955335 learned_addr learned my addr 172.21.15.131:0/2477955335 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:28.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36dbfff640 1 -- 172.21.15.131:0/2477955335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36e40b09c0 msgr2=0x7f36e4144a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:28.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36e8f53640 1 --2- 172.21.15.131:0/2477955335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36e40b09c0 0x7f36e4144a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:28.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36e9754640 1 --2- 172.21.15.131:0/2477955335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36e41574d0 0x7f36e41542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:28.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36dbfff640 1 --2- 172.21.15.131:0/2477955335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36e40b09c0 0x7f36e4144a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36dbfff640 1 -- 172.21.15.131:0/2477955335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36e41574d0 msgr2=0x7f36e41542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:28.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36dbfff640 1 --2- 172.21.15.131:0/2477955335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36e41574d0 0x7f36e41542d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:28.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36dbfff640 1 -- 172.21.15.131:0/2477955335 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36dc009820 con 0x7f36e40b2600 2023-09-12T14:55:28.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36dbfff640 1 --2- 172.21.15.131:0/2477955335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36e40b2600 0x7f36e4153d90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f36d400c450 tx=0x7f36d400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:28.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36d97fa640 1 -- 172.21.15.131:0/2477955335 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d4012830 con 0x7f36e40b2600 2023-09-12T14:55:28.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36d97fa640 1 -- 172.21.15.131:0/2477955335 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f36d4012aa0 con 0x7f36e40b2600 2023-09-12T14:55:28.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36eb1b7640 1 -- 172.21.15.131:0/2477955335 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e41f4cc0 con 0x7f36e40b2600 2023-09-12T14:55:28.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36d97fa640 1 -- 172.21.15.131:0/2477955335 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d4011380 con 0x7f36e40b2600 2023-09-12T14:55:28.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.960+0000 7f36eb1b7640 1 -- 172.21.15.131:0/2477955335 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e41f7a70 con 0x7f36e40b2600 2023-09-12T14:55:28.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.964+0000 7f36eb1b7640 1 -- 172.21.15.131:0/2477955335 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e41f7f90 con 0x7f36e40b2600 2023-09-12T14:55:28.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.964+0000 7f36d97fa640 1 -- 172.21.15.131:0/2477955335 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f36d40114e0 con 0x7f36e40b2600 2023-09-12T14:55:28.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.964+0000 7f36d97fa640 1 --2- 172.21.15.131:0/2477955335 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f36d0065b50 0x7f36d0068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:28.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.964+0000 7f36d97fa640 1 -- 172.21.15.131:0/2477955335 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36d40832c0 con 0x7f36e40b2600 2023-09-12T14:55:28.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.964+0000 7f36e8f53640 1 --2- 172.21.15.131:0/2477955335 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f36d0065b50 0x7f36d0068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:28.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.964+0000 7f36e8f53640 1 --2- 172.21.15.131:0/2477955335 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f36d0065b50 0x7f36d0068010 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f36e0007fd0 tx=0x7f36e0009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:28.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:28.968+0000 7f36d97fa640 1 -- 172.21.15.131:0/2477955335 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f36e41f7f90 con 0x7f36e40b2600 2023-09-12T14:55:29.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.120+0000 7f36eb1b7640 1 -- 172.21.15.131:0/2477955335 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f36e41f7f90 con 0x7f36e40b2600 2023-09-12T14:55:29.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.120+0000 7f36d97fa640 1 -- 172.21.15.131:0/2477955335 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f36e41f7f90 con 0x7f36e40b2600 2023-09-12T14:55:29.125 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-09-12T14:55:29.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.124+0000 7f36eb1b7640 1 -- 172.21.15.131:0/2477955335 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f36d0065b50 msgr2=0x7f36d0068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:29.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.124+0000 7f36eb1b7640 1 --2- 172.21.15.131:0/2477955335 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f36d0065b50 0x7f36d0068010 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f36e0007fd0 tx=0x7f36e0009040 comp rx=0 tx=0).stop 2023-09-12T14:55:29.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.124+0000 7f36eb1b7640 1 -- 172.21.15.131:0/2477955335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36e40b2600 msgr2=0x7f36e4153d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:29.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.124+0000 7f36eb1b7640 1 --2- 172.21.15.131:0/2477955335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36e40b2600 0x7f36e4153d90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f36d400c450 tx=0x7f36d400c910 comp rx=0 tx=0).stop 2023-09-12T14:55:29.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.124+0000 7f36eb1b7640 1 -- 172.21.15.131:0/2477955335 shutdown_connections 2023-09-12T14:55:29.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.124+0000 7f36eb1b7640 1 --2- 172.21.15.131:0/2477955335 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f36d0065b50 0x7f36d0068010 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:29.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.124+0000 7f36eb1b7640 1 --2- 172.21.15.131:0/2477955335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36e41574d0 0x7f36e41542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:29.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.124+0000 7f36eb1b7640 1 --2- 172.21.15.131:0/2477955335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36e40b2600 0x7f36e4153d90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:29.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.124+0000 7f36eb1b7640 1 --2- 172.21.15.131:0/2477955335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36e40b09c0 0x7f36e4144a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:29.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.124+0000 7f36eb1b7640 1 -- 172.21.15.131:0/2477955335 >> 172.21.15.131:0/2477955335 conn(0x7f36e4069840 msgr2=0x7f36e40b1380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:29.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.124+0000 7f36eb1b7640 1 -- 172.21.15.131:0/2477955335 shutdown_connections 2023-09-12T14:55:29.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.124+0000 7f36eb1b7640 1 -- 172.21.15.131:0/2477955335 wait complete. 2023-09-12T14:55:29.144 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T14:55:29.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7cf1ad640 1 -- 172.21.15.131:0/2407132669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7b80039d0 msgr2=0x7fd7b8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:29.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7cf1ad640 1 --2- 172.21.15.131:0/2407132669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7b80039d0 0x7fd7b8003dd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00bae0 tx=0x7fd7bc01ccf0 comp rx=0 tx=0).stop 2023-09-12T14:55:29.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7cf1ad640 1 -- 172.21.15.131:0/2407132669 shutdown_connections 2023-09-12T14:55:29.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7cf1ad640 1 --2- 172.21.15.131:0/2407132669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7b80039d0 0x7fd7b8003dd0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:29.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7cf1ad640 1 -- 172.21.15.131:0/2407132669 >> 172.21.15.131:0/2407132669 conn(0x7fd7c8069840 msgr2=0x7fd7c80cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:29.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7cf1ad640 1 -- 172.21.15.131:0/2407132669 shutdown_connections 2023-09-12T14:55:29.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7cf1ad640 1 -- 172.21.15.131:0/2407132669 wait complete. 2023-09-12T14:55:29.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7cf1ad640 1 Processor -- start 2023-09-12T14:55:29.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7cf1ad640 1 -- start start 2023-09-12T14:55:29.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7cf1ad640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7b80039d0 0x7fd7c80aa740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:29.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7cf1ad640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7c81e61c0 0x7fd7c80ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:29.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7cf1ad640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7c814f2a0 0x7fd7c814f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:29.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7cf1ad640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c80a68b0 con 0x7fd7c814f2a0 2023-09-12T14:55:29.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7cf1ad640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c80c90a0 con 0x7fd7b80039d0 2023-09-12T14:55:29.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7cf1ad640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c814fc60 con 0x7fd7c81e61c0 2023-09-12T14:55:29.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7cd74a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7c814f2a0 0x7fd7c814f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:29.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7cd74a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7c814f2a0 0x7fd7c814f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59756/0 (socket says 172.21.15.131:59756) 2023-09-12T14:55:29.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7cd74a640 1 -- 172.21.15.131:0/794315662 learned_addr learned my addr 172.21.15.131:0/794315662 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:29.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7c7fff640 1 --2- 172.21.15.131:0/794315662 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7c81e61c0 0x7fd7c80ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:29.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7ccf49640 1 --2- 172.21.15.131:0/794315662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7b80039d0 0x7fd7c80aa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:29.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7cd74a640 1 -- 172.21.15.131:0/794315662 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7c81e61c0 msgr2=0x7fd7c80ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:29.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7cd74a640 1 --2- 172.21.15.131:0/794315662 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7c81e61c0 0x7fd7c80ad7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:29.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7cd74a640 1 -- 172.21.15.131:0/794315662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7b80039d0 msgr2=0x7fd7c80aa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:29.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7cd74a640 1 --2- 172.21.15.131:0/794315662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7b80039d0 0x7fd7c80aa740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:29.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7cd74a640 1 -- 172.21.15.131:0/794315662 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c80aae40 con 0x7fd7c814f2a0 2023-09-12T14:55:29.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7c7fff640 1 --2- 172.21.15.131:0/794315662 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7c81e61c0 0x7fd7c80ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:55:29.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7cd74a640 1 --2- 172.21.15.131:0/794315662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7c814f2a0 0x7fd7c814f720 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80cd9c0 tx=0x7fd7c000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:29.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7c57fa640 1 -- 172.21.15.131:0/794315662 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0003d90 con 0x7fd7c814f2a0 2023-09-12T14:55:29.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7c57fa640 1 -- 172.21.15.131:0/794315662 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd7c0005070 con 0x7fd7c814f2a0 2023-09-12T14:55:29.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7cf1ad640 1 -- 172.21.15.131:0/794315662 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c80ab120 con 0x7fd7c814f2a0 2023-09-12T14:55:29.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7c57fa640 1 -- 172.21.15.131:0/794315662 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0005460 con 0x7fd7c814f2a0 2023-09-12T14:55:29.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7cf1ad640 1 -- 172.21.15.131:0/794315662 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c80a64d0 con 0x7fd7c814f2a0 2023-09-12T14:55:29.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.260+0000 7fd7cf1ad640 1 -- 172.21.15.131:0/794315662 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c80a69f0 con 0x7fd7c814f2a0 2023-09-12T14:55:29.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.264+0000 7fd7c57fa640 1 -- 172.21.15.131:0/794315662 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fd7c00055c0 con 0x7fd7c814f2a0 2023-09-12T14:55:29.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.264+0000 7fd7c57fa640 1 --2- 172.21.15.131:0/794315662 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd7ac065b20 0x7fd7ac067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:29.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.264+0000 7fd7ccf49640 1 --2- 172.21.15.131:0/794315662 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd7ac065b20 0x7fd7ac067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:29.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.264+0000 7fd7c57fa640 1 -- 172.21.15.131:0/794315662 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7c0082bd0 con 0x7fd7c814f2a0 2023-09-12T14:55:29.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.264+0000 7fd7ccf49640 1 --2- 172.21.15.131:0/794315662 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd7ac065b20 0x7fd7ac067fe0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0b0000 tx=0x7fd7bc0a7040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:29.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.268+0000 7fd7c57fa640 1 -- 172.21.15.131:0/794315662 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fd7c80a69f0 con 0x7fd7c814f2a0 2023-09-12T14:55:29.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.408+0000 7fd7cf1ad640 1 -- 172.21.15.131:0/794315662 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd7c80a69f0 con 0x7fd7c814f2a0 2023-09-12T14:55:29.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.408+0000 7fd7c57fa640 1 -- 172.21.15.131:0/794315662 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd7c80a69f0 con 0x7fd7c814f2a0 2023-09-12T14:55:29.413 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-09-12T14:55:29.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.412+0000 7fd7cf1ad640 1 -- 172.21.15.131:0/794315662 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd7ac065b20 msgr2=0x7fd7ac067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:29.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.412+0000 7fd7cf1ad640 1 --2- 172.21.15.131:0/794315662 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd7ac065b20 0x7fd7ac067fe0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0b0000 tx=0x7fd7bc0a7040 comp rx=0 tx=0).stop 2023-09-12T14:55:29.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.412+0000 7fd7cf1ad640 1 -- 172.21.15.131:0/794315662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7c814f2a0 msgr2=0x7fd7c814f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:29.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.412+0000 7fd7cf1ad640 1 --2- 172.21.15.131:0/794315662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7c814f2a0 0x7fd7c814f720 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80cd9c0 tx=0x7fd7c000ccd0 comp rx=0 tx=0).stop 2023-09-12T14:55:29.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.412+0000 7fd7cf1ad640 1 -- 172.21.15.131:0/794315662 shutdown_connections 2023-09-12T14:55:29.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.412+0000 7fd7cf1ad640 1 --2- 172.21.15.131:0/794315662 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd7ac065b20 0x7fd7ac067fe0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:29.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.412+0000 7fd7cf1ad640 1 --2- 172.21.15.131:0/794315662 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7c81e61c0 0x7fd7c80ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:29.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.412+0000 7fd7cf1ad640 1 --2- 172.21.15.131:0/794315662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7c814f2a0 0x7fd7c814f720 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:29.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.412+0000 7fd7cf1ad640 1 --2- 172.21.15.131:0/794315662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7b80039d0 0x7fd7c80aa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:29.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.412+0000 7fd7cf1ad640 1 -- 172.21.15.131:0/794315662 >> 172.21.15.131:0/794315662 conn(0x7fd7c8069840 msgr2=0x7fd7c806fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:29.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.412+0000 7fd7cf1ad640 1 -- 172.21.15.131:0/794315662 shutdown_connections 2023-09-12T14:55:29.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.412+0000 7fd7cf1ad640 1 -- 172.21.15.131:0/794315662 wait complete. 2023-09-12T14:55:29.430 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values' 2023-09-12T14:55:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd9cea640 1 -- 172.21.15.131:0/1048521404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcbc0047d0 msgr2=0x7fdcbc004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:29.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd9cea640 1 --2- 172.21.15.131:0/1048521404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcbc0047d0 0x7fdcbc004c50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4001680 tx=0x7fdcc401c730 comp rx=0 tx=0).stop 2023-09-12T14:55:29.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd9cea640 1 -- 172.21.15.131:0/1048521404 shutdown_connections 2023-09-12T14:55:29.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd9cea640 1 --2- 172.21.15.131:0/1048521404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcbc0047d0 0x7fdcbc004c50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:29.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd9cea640 1 -- 172.21.15.131:0/1048521404 >> 172.21.15.131:0/1048521404 conn(0x7fdcd409a210 msgr2=0x7fdcd40d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:29.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd9cea640 1 -- 172.21.15.131:0/1048521404 shutdown_connections 2023-09-12T14:55:29.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd9cea640 1 -- 172.21.15.131:0/1048521404 wait complete. 2023-09-12T14:55:29.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd9cea640 1 Processor -- start 2023-09-12T14:55:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd9cea640 1 -- start start 2023-09-12T14:55:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd9cea640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcbc0047d0 0x7fdcd41e95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd9cea640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdcd41f8270 0x7fdcd41e7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd9cea640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdcd41e7ca0 0x7fdcd40b4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:29.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd9cea640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd40b3a70 con 0x7fdcbc0047d0 2023-09-12T14:55:29.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd9cea640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd41f8ac0 con 0x7fdcd41e7ca0 2023-09-12T14:55:29.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd9cea640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd41f8da0 con 0x7fdcd41f8270 2023-09-12T14:55:29.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd37fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcbc0047d0 0x7fdcd41e95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:29.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd37fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcbc0047d0 0x7fdcd41e95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59774/0 (socket says 172.21.15.131:59774) 2023-09-12T14:55:29.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd37fe640 1 -- 172.21.15.131:0/1641056901 learned_addr learned my addr 172.21.15.131:0/1641056901 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:29.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd2ffd640 1 --2- 172.21.15.131:0/1641056901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdcd41f8270 0x7fdcd41e7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:29.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd3fff640 1 --2- 172.21.15.131:0/1641056901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdcd41e7ca0 0x7fdcd40b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:29.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd37fe640 1 -- 172.21.15.131:0/1641056901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdcd41f8270 msgr2=0x7fdcd41e7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:29.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd37fe640 1 --2- 172.21.15.131:0/1641056901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdcd41f8270 0x7fdcd41e7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:29.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd37fe640 1 -- 172.21.15.131:0/1641056901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdcd41e7ca0 msgr2=0x7fdcd40b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:29.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd37fe640 1 --2- 172.21.15.131:0/1641056901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdcd41e7ca0 0x7fdcd40b4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:29.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd37fe640 1 -- 172.21.15.131:0/1641056901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd41f9080 con 0x7fdcbc0047d0 2023-09-12T14:55:29.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd37fe640 1 --2- 172.21.15.131:0/1641056901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcbc0047d0 0x7fdcd41e95e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8011780 tx=0x7fdcc8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:29.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcbbfff640 1 -- 172.21.15.131:0/1641056901 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc800fd50 con 0x7fdcbc0047d0 2023-09-12T14:55:29.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcbbfff640 1 -- 172.21.15.131:0/1641056901 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdcc8011e90 con 0x7fdcbc0047d0 2023-09-12T14:55:29.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd9cea640 1 -- 172.21.15.131:0/1641056901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd40b3d10 con 0x7fdcbc0047d0 2023-09-12T14:55:29.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcbbfff640 1 -- 172.21.15.131:0/1641056901 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc80126b0 con 0x7fdcbc0047d0 2023-09-12T14:55:29.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.540+0000 7fdcd9cea640 1 -- 172.21.15.131:0/1641056901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd40b4770 con 0x7fdcbc0047d0 2023-09-12T14:55:29.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.544+0000 7fdcd9cea640 1 -- 172.21.15.131:0/1641056901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd4073580 con 0x7fdcbc0047d0 2023-09-12T14:55:29.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.544+0000 7fdcbbfff640 1 -- 172.21.15.131:0/1641056901 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fdcc8012810 con 0x7fdcbc0047d0 2023-09-12T14:55:29.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.544+0000 7fdcbbfff640 1 --2- 172.21.15.131:0/1641056901 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdcc0065b20 0x7fdcc0067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:29.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.544+0000 7fdcbbfff640 1 -- 172.21.15.131:0/1641056901 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdcc80840d0 con 0x7fdcbc0047d0 2023-09-12T14:55:29.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.544+0000 7fdcd2ffd640 1 --2- 172.21.15.131:0/1641056901 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdcc0065b20 0x7fdcc0067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:29.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.544+0000 7fdcd2ffd640 1 --2- 172.21.15.131:0/1641056901 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdcc0065b20 0x7fdcc0067fe0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc004750 tx=0x7fdcc40194d0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:29.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.548+0000 7fdcbbfff640 1 -- 172.21.15.131:0/1641056901 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fdcc8051ad0 con 0x7fdcbc0047d0 2023-09-12T14:55:29.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:29.752+0000 7fdcd9cea640 1 -- 172.21.15.131:0/1641056901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values"]} v 0) v1 -- 0x7fdcd409a730 con 0x7fdcbc0047d0 2023-09-12T14:55:30.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.048+0000 7fdcbbfff640 1 -- 172.21.15.131:0/1641056901 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values"]}]=0 v52) v1 ==== 163+0+0 (secure 0 0 0) 0x7fdcc80550f0 con 0x7fdcbc0047d0 2023-09-12T14:55:30.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.052+0000 7fdcd9cea640 1 -- 172.21.15.131:0/1641056901 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdcc0065b20 msgr2=0x7fdcc0067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:30.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.052+0000 7fdcd9cea640 1 --2- 172.21.15.131:0/1641056901 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdcc0065b20 0x7fdcc0067fe0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc004750 tx=0x7fdcc40194d0 comp rx=0 tx=0).stop 2023-09-12T14:55:30.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.052+0000 7fdcd9cea640 1 -- 172.21.15.131:0/1641056901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcbc0047d0 msgr2=0x7fdcd41e95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:30.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.052+0000 7fdcd9cea640 1 --2- 172.21.15.131:0/1641056901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcbc0047d0 0x7fdcd41e95e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8011780 tx=0x7fdcc8011c40 comp rx=0 tx=0).stop 2023-09-12T14:55:30.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.052+0000 7fdcd9cea640 1 -- 172.21.15.131:0/1641056901 shutdown_connections 2023-09-12T14:55:30.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.052+0000 7fdcd9cea640 1 --2- 172.21.15.131:0/1641056901 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdcc0065b20 0x7fdcc0067fe0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:30.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.052+0000 7fdcd9cea640 1 --2- 172.21.15.131:0/1641056901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdcd41f8270 0x7fdcd41e7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:30.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.052+0000 7fdcd9cea640 1 --2- 172.21.15.131:0/1641056901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdcd41e7ca0 0x7fdcd40b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:30.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.052+0000 7fdcd9cea640 1 --2- 172.21.15.131:0/1641056901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcbc0047d0 0x7fdcd41e95e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:30.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.052+0000 7fdcd9cea640 1 -- 172.21.15.131:0/1641056901 >> 172.21.15.131:0/1641056901 conn(0x7fdcd409a210 msgr2=0x7fdcd415e410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:30.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.052+0000 7fdcd9cea640 1 -- 172.21.15.131:0/1641056901 shutdown_connections 2023-09-12T14:55:30.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.052+0000 7fdcd9cea640 1 -- 172.21.15.131:0/1641056901 wait complete. 2023-09-12T14:55:30.075 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-09-12T14:55:30.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.192+0000 7f76048d5640 1 -- 172.21.15.131:0/3149283863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75e80047d0 msgr2=0x7f75e8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:30.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.192+0000 7f76048d5640 1 --2- 172.21.15.131:0/3149283863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75e80047d0 0x7f75e8004c50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f75f4001680 tx=0x7f75f401c760 comp rx=0 tx=0).stop 2023-09-12T14:55:30.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.192+0000 7f76048d5640 1 -- 172.21.15.131:0/3149283863 shutdown_connections 2023-09-12T14:55:30.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.192+0000 7f76048d5640 1 --2- 172.21.15.131:0/3149283863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75e80047d0 0x7f75e8004c50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:30.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.192+0000 7f76048d5640 1 -- 172.21.15.131:0/3149283863 >> 172.21.15.131:0/3149283863 conn(0x7f760009a200 msgr2=0x7f76000d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:30.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.192+0000 7f76048d5640 1 -- 172.21.15.131:0/3149283863 shutdown_connections 2023-09-12T14:55:30.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.192+0000 7f76048d5640 1 -- 172.21.15.131:0/3149283863 wait complete. 2023-09-12T14:55:30.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.192+0000 7f76048d5640 1 Processor -- start 2023-09-12T14:55:30.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.192+0000 7f76048d5640 1 -- start start 2023-09-12T14:55:30.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.192+0000 7f76048d5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75e80047d0 0x7f76000b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:30.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.192+0000 7f76048d5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76000b43c0 0x7f760014a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:30.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.192+0000 7f76048d5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7600146cb0 0x7f76000cd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:30.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.192+0000 7f76048d5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7600147740 con 0x7f75e80047d0 2023-09-12T14:55:30.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.192+0000 7f76048d5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76000b4820 con 0x7f76000b43c0 2023-09-12T14:55:30.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.192+0000 7f76048d5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76000b4a90 con 0x7f7600146cb0 2023-09-12T14:55:30.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.192+0000 7f75fe59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75e80047d0 0x7f76000b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:30.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.192+0000 7f75fe59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75e80047d0 0x7f76000b2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59786/0 (socket says 172.21.15.131:59786) 2023-09-12T14:55:30.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.192+0000 7f75fe59c640 1 -- 172.21.15.131:0/3166514553 learned_addr learned my addr 172.21.15.131:0/3166514553 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:30.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.192+0000 7f75fe59c640 1 -- 172.21.15.131:0/3166514553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7600146cb0 msgr2=0x7f76000cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:30.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.192+0000 7f75fe59c640 1 --2- 172.21.15.131:0/3166514553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7600146cb0 0x7f76000cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:30.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.192+0000 7f75fe59c640 1 -- 172.21.15.131:0/3166514553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76000b43c0 msgr2=0x7f760014a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:30.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.192+0000 7f75fe59c640 1 --2- 172.21.15.131:0/3166514553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76000b43c0 0x7f760014a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:30.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.192+0000 7f75fe59c640 1 -- 172.21.15.131:0/3166514553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f760014a6d0 con 0x7f75e80047d0 2023-09-12T14:55:30.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.192+0000 7f75fe59c640 1 --2- 172.21.15.131:0/3166514553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75e80047d0 0x7f76000b2be0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f75f0010990 tx=0x7f75f0010e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:30.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.196+0000 7f75eeffd640 1 -- 172.21.15.131:0/3166514553 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f0009730 con 0x7f75e80047d0 2023-09-12T14:55:30.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.196+0000 7f75eeffd640 1 -- 172.21.15.131:0/3166514553 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f75f000ed80 con 0x7f75e80047d0 2023-09-12T14:55:30.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.196+0000 7f75eeffd640 1 -- 172.21.15.131:0/3166514553 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f0005280 con 0x7f75e80047d0 2023-09-12T14:55:30.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.196+0000 7f76048d5640 1 -- 172.21.15.131:0/3166514553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7600147990 con 0x7f75e80047d0 2023-09-12T14:55:30.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.196+0000 7f76048d5640 1 -- 172.21.15.131:0/3166514553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7600147f50 con 0x7f75e80047d0 2023-09-12T14:55:30.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.196+0000 7f76048d5640 1 -- 172.21.15.131:0/3166514553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76000b26c0 con 0x7f75e80047d0 2023-09-12T14:55:30.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.196+0000 7f75eeffd640 1 -- 172.21.15.131:0/3166514553 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f75f00053e0 con 0x7f75e80047d0 2023-09-12T14:55:30.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.196+0000 7f75eeffd640 1 --2- 172.21.15.131:0/3166514553 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f75e0065b20 0x7f75e0067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:30.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.196+0000 7f75eeffd640 1 -- 172.21.15.131:0/3166514553 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75f0016070 con 0x7f75e80047d0 2023-09-12T14:55:30.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.196+0000 7f75fdd9b640 1 --2- 172.21.15.131:0/3166514553 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f75e0065b20 0x7f75e0067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:30.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.196+0000 7f75fdd9b640 1 --2- 172.21.15.131:0/3166514553 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f75e0065b20 0x7f75e0067fe0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f76000d1ac0 tx=0x7f75e8005800 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:30.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.200+0000 7f75eeffd640 1 -- 172.21.15.131:0/3166514553 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f75f0051f80 con 0x7f75e80047d0 2023-09-12T14:55:30.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.344+0000 7f76048d5640 1 -- 172.21.15.131:0/3166514553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7f7600073580 con 0x7f75e80047d0 2023-09-12T14:55:30.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.344+0000 7f75eeffd640 1 -- 172.21.15.131:0/3166514553 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v30) v1 ==== 59+0+2220 (secure 0 0 0) 0x7f75f00555a0 con 0x7f75e80047d0 2023-09-12T14:55:30.349 INFO:teuthology.orchestra.run.smithi131.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-09-12T14:55:30.349 INFO:teuthology.orchestra.run.smithi131.stdout:mon advanced mon_op_complaint_time 120 2023-09-12T14:55:30.349 INFO:teuthology.orchestra.run.smithi131.stdout:osd advanced osd_op_complaint_time 180.000000 2023-09-12T14:55:30.349 INFO:teuthology.orchestra.run.smithi131.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3490.021839 2023-09-12T14:55:30.349 INFO:teuthology.orchestra.run.smithi131.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3633.369832 2023-09-12T14:55:30.350 INFO:teuthology.orchestra.run.smithi131.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3738.104552 2023-09-12T14:55:30.350 INFO:teuthology.orchestra.run.smithi131.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3271.929947 2023-09-12T14:55:30.350 INFO:teuthology.orchestra.run.smithi131.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4466.851282 2023-09-12T14:55:30.350 INFO:teuthology.orchestra.run.smithi131.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4239.387573 2023-09-12T14:55:30.350 INFO:teuthology.orchestra.run.smithi131.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4119.423499 2023-09-12T14:55:30.351 INFO:teuthology.orchestra.run.smithi131.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3966.338174 2023-09-12T14:55:30.351 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced debug_mds 20/20 2023-09-12T14:55:30.351 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced debug_mds_balancer 20/20 2023-09-12T14:55:30.351 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced debug_ms 1/1 2023-09-12T14:55:30.351 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced mds_bal_fragment_size_max 10000 2023-09-12T14:55:30.352 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced mds_bal_merge_size 5 2023-09-12T14:55:30.352 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced mds_bal_split_bits 3 2023-09-12T14:55:30.352 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced mds_bal_split_size 100 2023-09-12T14:55:30.352 INFO:teuthology.orchestra.run.smithi131.stdout:mds dev mds_debug_frag true 2023-09-12T14:55:30.352 INFO:teuthology.orchestra.run.smithi131.stdout:mds dev mds_debug_scatterstat true 2023-09-12T14:55:30.353 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced mds_op_complaint_time 180.000000 2023-09-12T14:55:30.353 INFO:teuthology.orchestra.run.smithi131.stdout:mds dev mds_verify_scatter true 2023-09-12T14:55:30.353 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced osd_op_complaint_time 180.000000 2023-09-12T14:55:30.353 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced rados_mon_op_timeout 900 2023-09-12T14:55:30.353 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced rados_osd_op_timeout 900 2023-09-12T14:55:30.354 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced client_mount_timeout 600 2023-09-12T14:55:30.354 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced debug_client 20/20 2023-09-12T14:55:30.354 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced debug_ms 1/1 2023-09-12T14:55:30.354 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced rados_mon_op_timeout 900 2023-09-12T14:55:30.354 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced rados_osd_op_timeout 900 2023-09-12T14:55:30.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.348+0000 7f76048d5640 1 -- 172.21.15.131:0/3166514553 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f75e0065b20 msgr2=0x7f75e0067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:30.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.348+0000 7f76048d5640 1 --2- 172.21.15.131:0/3166514553 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f75e0065b20 0x7f75e0067fe0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f76000d1ac0 tx=0x7f75e8005800 comp rx=0 tx=0).stop 2023-09-12T14:55:30.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.348+0000 7f76048d5640 1 -- 172.21.15.131:0/3166514553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75e80047d0 msgr2=0x7f76000b2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:30.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.348+0000 7f76048d5640 1 --2- 172.21.15.131:0/3166514553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75e80047d0 0x7f76000b2be0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f75f0010990 tx=0x7f75f0010e50 comp rx=0 tx=0).stop 2023-09-12T14:55:30.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.348+0000 7f76048d5640 1 -- 172.21.15.131:0/3166514553 shutdown_connections 2023-09-12T14:55:30.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.348+0000 7f76048d5640 1 --2- 172.21.15.131:0/3166514553 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f75e0065b20 0x7f75e0067fe0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:30.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.348+0000 7f76048d5640 1 --2- 172.21.15.131:0/3166514553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7600146cb0 0x7f76000cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:30.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.348+0000 7f76048d5640 1 --2- 172.21.15.131:0/3166514553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76000b43c0 0x7f760014a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:30.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.348+0000 7f76048d5640 1 --2- 172.21.15.131:0/3166514553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75e80047d0 0x7f76000b2be0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:30.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.348+0000 7f76048d5640 1 -- 172.21.15.131:0/3166514553 >> 172.21.15.131:0/3166514553 conn(0x7f760009a200 msgr2=0x7f760015d710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:30.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.348+0000 7f76048d5640 1 -- 172.21.15.131:0/3166514553 shutdown_connections 2023-09-12T14:55:30.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.348+0000 7f76048d5640 1 -- 172.21.15.131:0/3166514553 wait complete. 2023-09-12T14:55:30.365 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-09-12T14:55:30.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01db97e640 1 -- 172.21.15.131:0/1704364598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01d406b050 msgr2=0x7f01d406b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:30.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01db97e640 1 --2- 172.21.15.131:0/1704364598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01d406b050 0x7f01d406b460 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f01d406eea0 tx=0x7f01cc01ccf0 comp rx=0 tx=0).stop 2023-09-12T14:55:30.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01db97e640 1 -- 172.21.15.131:0/1704364598 shutdown_connections 2023-09-12T14:55:30.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01db97e640 1 --2- 172.21.15.131:0/1704364598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01d406b050 0x7f01d406b460 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:30.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01db97e640 1 --2- 172.21.15.131:0/1704364598 >> v2:172.21.15.187:3300/0 conn(0x7f01d40ab730 0x7f01d40abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:30.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01db97e640 1 -- 172.21.15.131:0/1704364598 >> 172.21.15.131:0/1704364598 conn(0x7f01d4069840 msgr2=0x7f01d40cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:30.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01db97e640 1 -- 172.21.15.131:0/1704364598 shutdown_connections 2023-09-12T14:55:30.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01db97e640 1 -- 172.21.15.131:0/1704364598 wait complete. 2023-09-12T14:55:30.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01db97e640 1 Processor -- start 2023-09-12T14:55:30.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01db97e640 1 -- start start 2023-09-12T14:55:30.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01db97e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01d406b050 0x7f01d40b2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:30.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01db97e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01d40ab730 0x7f01d4148420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:30.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01db97e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01d4158180 0x7f01d40ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:30.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01db97e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d4148960 con 0x7f01d406b050 2023-09-12T14:55:30.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01db97e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d40b10e0 con 0x7f01d40ab730 2023-09-12T14:55:30.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01db97e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d40b13c0 con 0x7f01d4158180 2023-09-12T14:55:30.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01d971a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01d406b050 0x7f01d40b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:30.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01d971a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01d406b050 0x7f01d40b2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59804/0 (socket says 172.21.15.131:59804) 2023-09-12T14:55:30.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01d971a640 1 -- 172.21.15.131:0/2977978069 learned_addr learned my addr 172.21.15.131:0/2977978069 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:30.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01d971a640 1 -- 172.21.15.131:0/2977978069 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01d4158180 msgr2=0x7f01d40ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:30.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01d971a640 1 --2- 172.21.15.131:0/2977978069 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01d4158180 0x7f01d40ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:30.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01d971a640 1 -- 172.21.15.131:0/2977978069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01d40ab730 msgr2=0x7f01d4148420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:30.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01d971a640 1 --2- 172.21.15.131:0/2977978069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01d40ab730 0x7f01d4148420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:30.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01d971a640 1 -- 172.21.15.131:0/2977978069 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01cc0098e0 con 0x7f01d406b050 2023-09-12T14:55:30.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01d971a640 1 --2- 172.21.15.131:0/2977978069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01d406b050 0x7f01d40b2180 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f01d000b550 tx=0x7f01d000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:30.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01c9ffb640 1 -- 172.21.15.131:0/2977978069 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0010870 con 0x7f01d406b050 2023-09-12T14:55:30.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01c9ffb640 1 -- 172.21.15.131:0/2977978069 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f01d0010ae0 con 0x7f01d406b050 2023-09-12T14:55:30.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01db97e640 1 -- 172.21.15.131:0/2977978069 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d40b16a0 con 0x7f01d406b050 2023-09-12T14:55:30.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01c9ffb640 1 -- 172.21.15.131:0/2977978069 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d000f3c0 con 0x7f01d406b050 2023-09-12T14:55:30.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01db97e640 1 -- 172.21.15.131:0/2977978069 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d40b1c60 con 0x7f01d406b050 2023-09-12T14:55:30.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.480+0000 7f01db97e640 1 -- 172.21.15.131:0/2977978069 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d40b1e50 con 0x7f01d406b050 2023-09-12T14:55:30.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.484+0000 7f01c9ffb640 1 -- 172.21.15.131:0/2977978069 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f01d000f520 con 0x7f01d406b050 2023-09-12T14:55:30.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.484+0000 7f01c9ffb640 1 --2- 172.21.15.131:0/2977978069 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f01b8065b20 0x7f01b8067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:30.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.484+0000 7f01c9ffb640 1 -- 172.21.15.131:0/2977978069 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01d00835c0 con 0x7f01d406b050 2023-09-12T14:55:30.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.484+0000 7f01d8f19640 1 --2- 172.21.15.131:0/2977978069 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f01b8065b20 0x7f01b8067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:30.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.484+0000 7f01d8f19640 1 --2- 172.21.15.131:0/2977978069 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f01b8065b20 0x7f01b8067fe0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f01d40cb2c0 tx=0x7f01c4006bb0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:30.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.488+0000 7f01c9ffb640 1 -- 172.21.15.131:0/2977978069 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f01d0050f10 con 0x7f01d406b050 2023-09-12T14:55:30.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.628+0000 7f01db97e640 1 -- 172.21.15.131:0/2977978069 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f01d4073580 con 0x7f01d406b050 2023-09-12T14:55:30.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.636+0000 7f01c9ffb640 1 -- 172.21.15.131:0/2977978069 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v31)=0 v31) v1 ==== 122+0+0 (secure 0 0 0) 0x7f01d0054530 con 0x7f01d406b050 2023-09-12T14:55:30.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.636+0000 7f01db97e640 1 -- 172.21.15.131:0/2977978069 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f01b8065b20 msgr2=0x7f01b8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:30.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.636+0000 7f01db97e640 1 --2- 172.21.15.131:0/2977978069 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f01b8065b20 0x7f01b8067fe0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f01d40cb2c0 tx=0x7f01c4006bb0 comp rx=0 tx=0).stop 2023-09-12T14:55:30.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.636+0000 7f01db97e640 1 -- 172.21.15.131:0/2977978069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01d406b050 msgr2=0x7f01d40b2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:30.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.636+0000 7f01db97e640 1 --2- 172.21.15.131:0/2977978069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01d406b050 0x7f01d40b2180 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f01d000b550 tx=0x7f01d000ba10 comp rx=0 tx=0).stop 2023-09-12T14:55:30.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.636+0000 7f01db97e640 1 -- 172.21.15.131:0/2977978069 shutdown_connections 2023-09-12T14:55:30.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.636+0000 7f01db97e640 1 --2- 172.21.15.131:0/2977978069 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f01b8065b20 0x7f01b8067fe0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:30.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.636+0000 7f01db97e640 1 --2- 172.21.15.131:0/2977978069 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01d4158180 0x7f01d40ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:30.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.636+0000 7f01db97e640 1 --2- 172.21.15.131:0/2977978069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01d40ab730 0x7f01d4148420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:30.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.636+0000 7f01db97e640 1 --2- 172.21.15.131:0/2977978069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01d406b050 0x7f01d40b2180 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:30.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.636+0000 7f01db97e640 1 -- 172.21.15.131:0/2977978069 >> 172.21.15.131:0/2977978069 conn(0x7f01d4069840 msgr2=0x7f01d40ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:30.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.636+0000 7f01db97e640 1 -- 172.21.15.131:0/2977978069 shutdown_connections 2023-09-12T14:55:30.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.636+0000 7f01db97e640 1 -- 172.21.15.131:0/2977978069 wait complete. 2023-09-12T14:55:30.656 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-12T14:55:30.656 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-12T14:55:30.715 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-12T14:55:30.760 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-12T14:55:30.761 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:55:30.761 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-12T14:55:30.766 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:55:30.767 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-12T14:55:30.814 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:55:30.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.984+0000 7f319090a640 1 -- 172.21.15.131:0/1873811834 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f318c0d1680 msgr2=0x7f318c0b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:30.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.984+0000 7f319090a640 1 --2- 172.21.15.131:0/1873811834 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f318c0d1680 0x7f318c0b0750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3180009930 tx=0x7f318001c890 comp rx=0 tx=0).stop 2023-09-12T14:55:30.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.984+0000 7f319090a640 1 -- 172.21.15.131:0/1873811834 shutdown_connections 2023-09-12T14:55:30.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.984+0000 7f319090a640 1 --2- 172.21.15.131:0/1873811834 >> v2:172.21.15.187:3300/0 conn(0x7f318c15e9f0 0x7f318c0b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:30.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.984+0000 7f319090a640 1 --2- 172.21.15.131:0/1873811834 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f318c0d1680 0x7f318c0b0750 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:30.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.984+0000 7f319090a640 1 -- 172.21.15.131:0/1873811834 >> 172.21.15.131:0/1873811834 conn(0x7f318c09a200 msgr2=0x7f318c0d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:30.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.984+0000 7f319090a640 1 -- 172.21.15.131:0/1873811834 shutdown_connections 2023-09-12T14:55:30.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.984+0000 7f319090a640 1 -- 172.21.15.131:0/1873811834 wait complete. 2023-09-12T14:55:30.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.988+0000 7f319090a640 1 Processor -- start 2023-09-12T14:55:30.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.988+0000 7f319090a640 1 -- start start 2023-09-12T14:55:30.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.988+0000 7f319090a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f318c0cf2d0 0x7f318c15aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:30.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.988+0000 7f319090a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f318c0d1680 0x7f318c1533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:30.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.988+0000 7f319090a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f318c15e9f0 0x7f318c153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:30.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.988+0000 7f319090a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c154b70 con 0x7f318c0cf2d0 2023-09-12T14:55:30.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.988+0000 7f319090a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c1529c0 con 0x7f318c15e9f0 2023-09-12T14:55:30.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.988+0000 7f319090a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f318c153f60 con 0x7f318c0d1680 2023-09-12T14:55:30.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.988+0000 7f318a59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f318c0cf2d0 0x7f318c15aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:30.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.988+0000 7f318a59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f318c0cf2d0 0x7f318c15aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59810/0 (socket says 172.21.15.131:59810) 2023-09-12T14:55:30.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.988+0000 7f318a59c640 1 -- 172.21.15.131:0/961124644 learned_addr learned my addr 172.21.15.131:0/961124644 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:30.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.988+0000 7f318ad9d640 1 --2- 172.21.15.131:0/961124644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f318c15e9f0 0x7f318c153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:30.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.988+0000 7f3189d9b640 1 --2- 172.21.15.131:0/961124644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f318c0d1680 0x7f318c1533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:30.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.988+0000 7f318a59c640 1 -- 172.21.15.131:0/961124644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f318c0d1680 msgr2=0x7f318c1533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:30.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.988+0000 7f318a59c640 1 --2- 172.21.15.131:0/961124644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f318c0d1680 0x7f318c1533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:30.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.988+0000 7f318a59c640 1 -- 172.21.15.131:0/961124644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f318c15e9f0 msgr2=0x7f318c153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:30.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.988+0000 7f318a59c640 1 --2- 172.21.15.131:0/961124644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f318c15e9f0 0x7f318c153930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:30.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.988+0000 7f318a59c640 1 -- 172.21.15.131:0/961124644 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3180009560 con 0x7f318c0cf2d0 2023-09-12T14:55:30.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.988+0000 7f318a59c640 1 --2- 172.21.15.131:0/961124644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f318c0cf2d0 0x7f318c15aa00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f317c00c420 tx=0x7f317c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:30.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.988+0000 7f317affd640 1 -- 172.21.15.131:0/961124644 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c0052d0 con 0x7f318c0cf2d0 2023-09-12T14:55:30.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.988+0000 7f317affd640 1 -- 172.21.15.131:0/961124644 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f317c007020 con 0x7f318c0cf2d0 2023-09-12T14:55:30.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.988+0000 7f319090a640 1 -- 172.21.15.131:0/961124644 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f318c154240 con 0x7f318c0cf2d0 2023-09-12T14:55:30.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.988+0000 7f317affd640 1 -- 172.21.15.131:0/961124644 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c007430 con 0x7f318c0cf2d0 2023-09-12T14:55:30.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.988+0000 7f319090a640 1 -- 172.21.15.131:0/961124644 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f318c154800 con 0x7f318c0cf2d0 2023-09-12T14:55:30.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.988+0000 7f319090a640 1 -- 172.21.15.131:0/961124644 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f318c073580 con 0x7f318c0cf2d0 2023-09-12T14:55:30.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.988+0000 7f317affd640 1 -- 172.21.15.131:0/961124644 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f317c0067e0 con 0x7f318c0cf2d0 2023-09-12T14:55:30.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.988+0000 7f317affd640 1 --2- 172.21.15.131:0/961124644 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f316c065b20 0x7f316c067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:30.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.992+0000 7f3189d9b640 1 --2- 172.21.15.131:0/961124644 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f316c065b20 0x7f316c067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:30.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.992+0000 7f317affd640 1 -- 172.21.15.131:0/961124644 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f317c0829f0 con 0x7f318c0cf2d0 2023-09-12T14:55:30.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.992+0000 7f3189d9b640 1 --2- 172.21.15.131:0/961124644 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f316c065b20 0x7f316c067fe0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f31800af000 tx=0x7f318001b0d0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:30.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:30.996+0000 7f317affd640 1 -- 172.21.15.131:0/961124644 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f317c0502b0 con 0x7f318c0cf2d0 2023-09-12T14:55:31.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.148+0000 7f319090a640 1 -- 172.21.15.131:0/961124644 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f318c0affd0 con 0x7f318c0cf2d0 2023-09-12T14:55:31.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.148+0000 7f317affd640 1 -- 172.21.15.131:0/961124644 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f317c0538d0 con 0x7f318c0cf2d0 2023-09-12T14:55:31.154 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:31.154 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:54:59.694778+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:54:55.689983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T14:54:57.029447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:55:31.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.152+0000 7f319090a640 1 -- 172.21.15.131:0/961124644 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f316c065b20 msgr2=0x7f316c067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:31.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.152+0000 7f319090a640 1 --2- 172.21.15.131:0/961124644 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f316c065b20 0x7f316c067fe0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f31800af000 tx=0x7f318001b0d0 comp rx=0 tx=0).stop 2023-09-12T14:55:31.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.152+0000 7f319090a640 1 -- 172.21.15.131:0/961124644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f318c0cf2d0 msgr2=0x7f318c15aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:31.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.152+0000 7f319090a640 1 --2- 172.21.15.131:0/961124644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f318c0cf2d0 0x7f318c15aa00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f317c00c420 tx=0x7f317c00c8e0 comp rx=0 tx=0).stop 2023-09-12T14:55:31.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.152+0000 7f319090a640 1 -- 172.21.15.131:0/961124644 shutdown_connections 2023-09-12T14:55:31.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.152+0000 7f319090a640 1 --2- 172.21.15.131:0/961124644 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f316c065b20 0x7f316c067fe0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:31.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.152+0000 7f319090a640 1 --2- 172.21.15.131:0/961124644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f318c0d1680 0x7f318c1533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:31.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.152+0000 7f319090a640 1 --2- 172.21.15.131:0/961124644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f318c15e9f0 0x7f318c153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:31.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.152+0000 7f319090a640 1 --2- 172.21.15.131:0/961124644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f318c0cf2d0 0x7f318c15aa00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:31.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.152+0000 7f319090a640 1 -- 172.21.15.131:0/961124644 >> 172.21.15.131:0/961124644 conn(0x7f318c09a200 msgr2=0x7f318c159a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:31.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.152+0000 7f319090a640 1 -- 172.21.15.131:0/961124644 shutdown_connections 2023-09-12T14:55:31.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.152+0000 7f319090a640 1 -- 172.21.15.131:0/961124644 wait complete. 2023-09-12T14:55:31.171 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T14:54:45.654004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T14:54:55.689983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T14:54:57.029447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-12T14:55:31.172 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T14:55:31.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.296+0000 7f255515c640 1 -- 172.21.15.131:0/660973794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f253c0052b0 msgr2=0x7f253c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:31.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.296+0000 7f255515c640 1 --2- 172.21.15.131:0/660973794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f253c0052b0 0x7f253c00b990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f25500ac0e0 tx=0x7f254801d1d0 comp rx=0 tx=0).stop 2023-09-12T14:55:31.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.296+0000 7f255515c640 1 -- 172.21.15.131:0/660973794 shutdown_connections 2023-09-12T14:55:31.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.296+0000 7f255515c640 1 --2- 172.21.15.131:0/660973794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f253c0052b0 0x7f253c00b990 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:31.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.296+0000 7f255515c640 1 -- 172.21.15.131:0/660973794 >> 172.21.15.131:0/660973794 conn(0x7f2550069840 msgr2=0x7f25500cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:31.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.296+0000 7f255515c640 1 -- 172.21.15.131:0/660973794 shutdown_connections 2023-09-12T14:55:31.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.296+0000 7f255515c640 1 -- 172.21.15.131:0/660973794 wait complete. 2023-09-12T14:55:31.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.296+0000 7f255515c640 1 Processor -- start 2023-09-12T14:55:31.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.296+0000 7f255515c640 1 -- start start 2023-09-12T14:55:31.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.296+0000 7f255515c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f253c0052b0 0x7f25501505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:31.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.296+0000 7f255515c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f255013ede0 0x7f25500b1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:31.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.296+0000 7f255515c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25500b1bc0 0x7f2550151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:31.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.296+0000 7f255515c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25500b2070 con 0x7f255013ede0 2023-09-12T14:55:31.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.296+0000 7f255515c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25501524b0 con 0x7f25500b1bc0 2023-09-12T14:55:31.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.296+0000 7f255515c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255014e090 con 0x7f253c0052b0 2023-09-12T14:55:31.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.296+0000 7f2547fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f255013ede0 0x7f25500b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:31.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.296+0000 7f2547fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f255013ede0 0x7f25500b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59838/0 (socket says 172.21.15.131:59838) 2023-09-12T14:55:31.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.296+0000 7f2547fff640 1 -- 172.21.15.131:0/655511115 learned_addr learned my addr 172.21.15.131:0/655511115 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:31.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.296+0000 7f254ed9d640 1 --2- 172.21.15.131:0/655511115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f253c0052b0 0x7f25501505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:31.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.296+0000 7f254f59e640 1 --2- 172.21.15.131:0/655511115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25500b1bc0 0x7f2550151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:31.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.296+0000 7f2547fff640 1 -- 172.21.15.131:0/655511115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f253c0052b0 msgr2=0x7f25501505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:31.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.296+0000 7f2547fff640 1 --2- 172.21.15.131:0/655511115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f253c0052b0 0x7f25501505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:31.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.296+0000 7f2547fff640 1 -- 172.21.15.131:0/655511115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25500b1bc0 msgr2=0x7f2550151780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:31.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.296+0000 7f2547fff640 1 --2- 172.21.15.131:0/655511115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25500b1bc0 0x7f2550151780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:31.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.296+0000 7f2547fff640 1 -- 172.21.15.131:0/655511115 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255014e370 con 0x7f255013ede0 2023-09-12T14:55:31.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.296+0000 7f254f59e640 1 --2- 172.21.15.131:0/655511115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25500b1bc0 0x7f2550151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:31.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.296+0000 7f2547fff640 1 --2- 172.21.15.131:0/655511115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f255013ede0 0x7f25500b1680 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2538004d80 tx=0x7f2538001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:31.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.300+0000 7f25477fe640 1 -- 172.21.15.131:0/655511115 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25380105d0 con 0x7f255013ede0 2023-09-12T14:55:31.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.300+0000 7f25477fe640 1 -- 172.21.15.131:0/655511115 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2538010820 con 0x7f255013ede0 2023-09-12T14:55:31.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.300+0000 7f255515c640 1 -- 172.21.15.131:0/655511115 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f255014e650 con 0x7f255013ede0 2023-09-12T14:55:31.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.300+0000 7f25477fe640 1 -- 172.21.15.131:0/655511115 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2538010c30 con 0x7f255013ede0 2023-09-12T14:55:31.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.300+0000 7f255515c640 1 -- 172.21.15.131:0/655511115 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2550150b10 con 0x7f255013ede0 2023-09-12T14:55:31.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.300+0000 7f255515c640 1 -- 172.21.15.131:0/655511115 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2550073580 con 0x7f255013ede0 2023-09-12T14:55:31.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.300+0000 7f25477fe640 1 -- 172.21.15.131:0/655511115 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f253801b460 con 0x7f255013ede0 2023-09-12T14:55:31.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.300+0000 7f25477fe640 1 --2- 172.21.15.131:0/655511115 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2530065b20 0x7f2530067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:31.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.300+0000 7f25477fe640 1 -- 172.21.15.131:0/655511115 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2538083c20 con 0x7f255013ede0 2023-09-12T14:55:31.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.300+0000 7f254ed9d640 1 --2- 172.21.15.131:0/655511115 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2530065b20 0x7f2530067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:31.310 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.300+0000 7f254ed9d640 1 --2- 172.21.15.131:0/655511115 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2530065b20 0x7f2530067fe0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f25400018f0 tx=0x7f25400014a0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:31.310 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.304+0000 7f25477fe640 1 -- 172.21.15.131:0/655511115 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f2538051600 con 0x7f255013ede0 2023-09-12T14:55:31.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.444+0000 7f255515c640 1 -- 172.21.15.131:0/655511115 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2550154fb0 con 0x7f255013ede0 2023-09-12T14:55:31.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.448+0000 7f25477fe640 1 -- 172.21.15.131:0/655511115 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2538054c20 con 0x7f255013ede0 2023-09-12T14:55:31.450 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-09-12T14:55:31.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.448+0000 7f255515c640 1 -- 172.21.15.131:0/655511115 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2530065b20 msgr2=0x7f2530067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:31.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.448+0000 7f255515c640 1 --2- 172.21.15.131:0/655511115 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2530065b20 0x7f2530067fe0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f25400018f0 tx=0x7f25400014a0 comp rx=0 tx=0).stop 2023-09-12T14:55:31.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.448+0000 7f255515c640 1 -- 172.21.15.131:0/655511115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f255013ede0 msgr2=0x7f25500b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:31.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.448+0000 7f255515c640 1 --2- 172.21.15.131:0/655511115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f255013ede0 0x7f25500b1680 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2538004d80 tx=0x7f2538001640 comp rx=0 tx=0).stop 2023-09-12T14:55:31.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.448+0000 7f255515c640 1 -- 172.21.15.131:0/655511115 shutdown_connections 2023-09-12T14:55:31.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.448+0000 7f255515c640 1 --2- 172.21.15.131:0/655511115 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2530065b20 0x7f2530067fe0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:31.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.448+0000 7f255515c640 1 --2- 172.21.15.131:0/655511115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25500b1bc0 0x7f2550151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:31.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.448+0000 7f255515c640 1 --2- 172.21.15.131:0/655511115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f255013ede0 0x7f25500b1680 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:31.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.448+0000 7f255515c640 1 --2- 172.21.15.131:0/655511115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f253c0052b0 0x7f25501505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:31.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.448+0000 7f255515c640 1 -- 172.21.15.131:0/655511115 >> 172.21.15.131:0/655511115 conn(0x7f2550069840 msgr2=0x7f255014d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:31.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.448+0000 7f255515c640 1 -- 172.21.15.131:0/655511115 shutdown_connections 2023-09-12T14:55:31.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.448+0000 7f255515c640 1 -- 172.21.15.131:0/655511115 wait complete. 2023-09-12T14:55:31.467 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T14:55:31.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.580+0000 7f04b1a3c640 1 -- 172.21.15.131:0/2916011097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0494005190 msgr2=0x7f049400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:31.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.580+0000 7f04b1a3c640 1 --2- 172.21.15.131:0/2916011097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0494005190 0x7f049400b990 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f04a400c5c0 tx=0x7f04a401d340 comp rx=0 tx=0).stop 2023-09-12T14:55:31.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.580+0000 7f04b1a3c640 1 -- 172.21.15.131:0/2916011097 shutdown_connections 2023-09-12T14:55:31.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.580+0000 7f04b1a3c640 1 --2- 172.21.15.131:0/2916011097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0494005190 0x7f049400b990 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:31.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.580+0000 7f04b1a3c640 1 -- 172.21.15.131:0/2916011097 >> 172.21.15.131:0/2916011097 conn(0x7f04ac09a200 msgr2=0x7f04ac0d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:31.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.580+0000 7f04b1a3c640 1 -- 172.21.15.131:0/2916011097 shutdown_connections 2023-09-12T14:55:31.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.580+0000 7f04b1a3c640 1 -- 172.21.15.131:0/2916011097 wait complete. 2023-09-12T14:55:31.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.580+0000 7f04b1a3c640 1 Processor -- start 2023-09-12T14:55:31.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.580+0000 7f04b1a3c640 1 -- start start 2023-09-12T14:55:31.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.580+0000 7f04b1a3c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0494005190 0x7f04ac156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:31.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.580+0000 7f04b1a3c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04ac0ca810 0x7f04ac0cac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:31.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.580+0000 7f04b1a3c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04ac153310 0x7f04ac1537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:31.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.580+0000 7f04b1a3c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac0cb4b0 con 0x7f04ac153310 2023-09-12T14:55:31.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.580+0000 7f04b1a3c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac1f44f0 con 0x7f0494005190 2023-09-12T14:55:31.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.580+0000 7f04b1a3c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac1f4790 con 0x7f04ac0ca810 2023-09-12T14:55:31.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.584+0000 7f04ab7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04ac153310 0x7f04ac1537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:31.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.584+0000 7f04ab7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04ac153310 0x7f04ac1537a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59866/0 (socket says 172.21.15.131:59866) 2023-09-12T14:55:31.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.584+0000 7f04ab7fe640 1 -- 172.21.15.131:0/4164712445 learned_addr learned my addr 172.21.15.131:0/4164712445 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:31.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.584+0000 7f04aa7fc640 1 --2- 172.21.15.131:0/4164712445 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04ac0ca810 0x7f04ac0cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:31.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.584+0000 7f04aaffd640 1 --2- 172.21.15.131:0/4164712445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0494005190 0x7f04ac156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:31.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.584+0000 7f04ab7fe640 1 -- 172.21.15.131:0/4164712445 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04ac0ca810 msgr2=0x7f04ac0cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:31.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.584+0000 7f04ab7fe640 1 --2- 172.21.15.131:0/4164712445 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04ac0ca810 0x7f04ac0cac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:31.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.584+0000 7f04ab7fe640 1 -- 172.21.15.131:0/4164712445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0494005190 msgr2=0x7f04ac156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:31.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.584+0000 7f04ab7fe640 1 --2- 172.21.15.131:0/4164712445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0494005190 0x7f04ac156b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:31.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.584+0000 7f04ab7fe640 1 -- 172.21.15.131:0/4164712445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ac1f4a70 con 0x7f04ac153310 2023-09-12T14:55:31.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.584+0000 7f04aaffd640 1 --2- 172.21.15.131:0/4164712445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0494005190 0x7f04ac156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:31.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.584+0000 7f04ab7fe640 1 --2- 172.21.15.131:0/4164712445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04ac153310 0x7f04ac1537a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f04a401dbe0 tx=0x7f04a4001310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:31.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.584+0000 7f0493fff640 1 -- 172.21.15.131:0/4164712445 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a40b5070 con 0x7f04ac153310 2023-09-12T14:55:31.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.584+0000 7f0493fff640 1 -- 172.21.15.131:0/4164712445 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f04a4001900 con 0x7f04ac153310 2023-09-12T14:55:31.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.584+0000 7f04b1a3c640 1 -- 172.21.15.131:0/4164712445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04ac1f4d50 con 0x7f04ac153310 2023-09-12T14:55:31.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.584+0000 7f0493fff640 1 -- 172.21.15.131:0/4164712445 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a4003be0 con 0x7f04ac153310 2023-09-12T14:55:31.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.584+0000 7f04b1a3c640 1 -- 172.21.15.131:0/4164712445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04ac1f5250 con 0x7f04ac153310 2023-09-12T14:55:31.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.584+0000 7f04b1a3c640 1 -- 172.21.15.131:0/4164712445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04ac1f4ee0 con 0x7f04ac153310 2023-09-12T14:55:31.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.584+0000 7f0493fff640 1 -- 172.21.15.131:0/4164712445 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f04a40ab030 con 0x7f04ac153310 2023-09-12T14:55:31.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.584+0000 7f0493fff640 1 --2- 172.21.15.131:0/4164712445 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0498065b20 0x7f0498067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:31.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.584+0000 7f0493fff640 1 -- 172.21.15.131:0/4164712445 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04a411c600 con 0x7f04ac153310 2023-09-12T14:55:31.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.584+0000 7f04aaffd640 1 --2- 172.21.15.131:0/4164712445 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0498065b20 0x7f0498067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:31.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.584+0000 7f04aaffd640 1 --2- 172.21.15.131:0/4164712445 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0498065b20 0x7f0498067fe0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f04ac0cbd90 tx=0x7f049c009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:31.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.588+0000 7f0493fff640 1 -- 172.21.15.131:0/4164712445 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f04a40e9f60 con 0x7f04ac153310 2023-09-12T14:55:31.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.732+0000 7f04b1a3c640 1 -- 172.21.15.131:0/4164712445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f04ac073580 con 0x7f04ac153310 2023-09-12T14:55:31.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.732+0000 7f0493fff640 1 -- 172.21.15.131:0/4164712445 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f04a40ed580 con 0x7f04ac153310 2023-09-12T14:55:31.735 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-09-12T14:55:31.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.732+0000 7f04b1a3c640 1 -- 172.21.15.131:0/4164712445 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0498065b20 msgr2=0x7f0498067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:31.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.732+0000 7f04b1a3c640 1 --2- 172.21.15.131:0/4164712445 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0498065b20 0x7f0498067fe0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f04ac0cbd90 tx=0x7f049c009040 comp rx=0 tx=0).stop 2023-09-12T14:55:31.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.732+0000 7f04b1a3c640 1 -- 172.21.15.131:0/4164712445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04ac153310 msgr2=0x7f04ac1537a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:31.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.732+0000 7f04b1a3c640 1 --2- 172.21.15.131:0/4164712445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04ac153310 0x7f04ac1537a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f04a401dbe0 tx=0x7f04a4001310 comp rx=0 tx=0).stop 2023-09-12T14:55:31.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.732+0000 7f04b1a3c640 1 -- 172.21.15.131:0/4164712445 shutdown_connections 2023-09-12T14:55:31.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.732+0000 7f04b1a3c640 1 --2- 172.21.15.131:0/4164712445 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0498065b20 0x7f0498067fe0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:31.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.732+0000 7f04b1a3c640 1 --2- 172.21.15.131:0/4164712445 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04ac0ca810 0x7f04ac0cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:31.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.732+0000 7f04b1a3c640 1 --2- 172.21.15.131:0/4164712445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04ac153310 0x7f04ac1537a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:31.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.732+0000 7f04b1a3c640 1 --2- 172.21.15.131:0/4164712445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0494005190 0x7f04ac156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:31.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.732+0000 7f04b1a3c640 1 -- 172.21.15.131:0/4164712445 >> 172.21.15.131:0/4164712445 conn(0x7f04ac09a200 msgr2=0x7f04ac0b0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:31.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.732+0000 7f04b1a3c640 1 -- 172.21.15.131:0/4164712445 shutdown_connections 2023-09-12T14:55:31.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.732+0000 7f04b1a3c640 1 -- 172.21.15.131:0/4164712445 wait complete. 2023-09-12T14:55:31.752 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T14:55:31.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.880+0000 7fe0167a6640 1 -- 172.21.15.131:0/2677638345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdffc0047d0 msgr2=0x7fdffc004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:31.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.880+0000 7fe0167a6640 1 --2- 172.21.15.131:0/2677638345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdffc0047d0 0x7fdffc004c50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdff8009b30 tx=0x7fdff801d210 comp rx=0 tx=0).stop 2023-09-12T14:55:31.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.880+0000 7fe0167a6640 1 -- 172.21.15.131:0/2677638345 shutdown_connections 2023-09-12T14:55:31.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.880+0000 7fe0167a6640 1 --2- 172.21.15.131:0/2677638345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdffc0047d0 0x7fdffc004c50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:31.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.880+0000 7fe0167a6640 1 -- 172.21.15.131:0/2677638345 >> 172.21.15.131:0/2677638345 conn(0x7fe010069840 msgr2=0x7fe0100cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:31.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.880+0000 7fe0167a6640 1 -- 172.21.15.131:0/2677638345 shutdown_connections 2023-09-12T14:55:31.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.880+0000 7fe0167a6640 1 -- 172.21.15.131:0/2677638345 wait complete. 2023-09-12T14:55:31.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.880+0000 7fe0167a6640 1 Processor -- start 2023-09-12T14:55:31.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.880+0000 7fe0167a6640 1 -- start start 2023-09-12T14:55:31.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.880+0000 7fe0167a6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdffc0047d0 0x7fe0101f4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:31.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.880+0000 7fe0167a6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe01015aee0 0x7fe0100ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:31.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.880+0000 7fe0167a6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0101e1350 0x7fe0101e5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:31.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.880+0000 7fe0167a6640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0101f5230 con 0x7fdffc0047d0 2023-09-12T14:55:31.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.880+0000 7fe0167a6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0101f5510 con 0x7fe01015aee0 2023-09-12T14:55:31.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.880+0000 7fe0167a6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0101e5790 con 0x7fe0101e1350 2023-09-12T14:55:31.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.880+0000 7fe00ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdffc0047d0 0x7fe0101f4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:31.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.880+0000 7fe00ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdffc0047d0 0x7fe0101f4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59898/0 (socket says 172.21.15.131:59898) 2023-09-12T14:55:31.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.880+0000 7fe00ffff640 1 -- 172.21.15.131:0/4046481747 learned_addr learned my addr 172.21.15.131:0/4046481747 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:31.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.884+0000 7fe00f7fe640 1 --2- 172.21.15.131:0/4046481747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe01015aee0 0x7fe0100ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:31.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.884+0000 7fe014d43640 1 --2- 172.21.15.131:0/4046481747 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0101e1350 0x7fe0101e5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:31.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.884+0000 7fe00ffff640 1 -- 172.21.15.131:0/4046481747 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0101e1350 msgr2=0x7fe0101e5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:31.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.884+0000 7fe00ffff640 1 --2- 172.21.15.131:0/4046481747 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0101e1350 0x7fe0101e5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:31.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.884+0000 7fe00ffff640 1 -- 172.21.15.131:0/4046481747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe01015aee0 msgr2=0x7fe0100ac6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:31.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.884+0000 7fe00ffff640 1 --2- 172.21.15.131:0/4046481747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe01015aee0 0x7fe0100ac6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:31.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.884+0000 7fe00ffff640 1 -- 172.21.15.131:0/4046481747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0101e5a70 con 0x7fdffc0047d0 2023-09-12T14:55:31.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.884+0000 7fe00f7fe640 1 --2- 172.21.15.131:0/4046481747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe01015aee0 0x7fe0100ac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:31.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.884+0000 7fe014d43640 1 --2- 172.21.15.131:0/4046481747 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0101e1350 0x7fe0101e5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:31.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.884+0000 7fe00ffff640 1 --2- 172.21.15.131:0/4046481747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdffc0047d0 0x7fe0101f4930 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe00800f630 tx=0x7fe00800faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:31.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.884+0000 7fe00cff9640 1 -- 172.21.15.131:0/4046481747 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00800a920 con 0x7fdffc0047d0 2023-09-12T14:55:31.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.884+0000 7fe00cff9640 1 -- 172.21.15.131:0/4046481747 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe00800ab70 con 0x7fdffc0047d0 2023-09-12T14:55:31.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.884+0000 7fe0167a6640 1 -- 172.21.15.131:0/4046481747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0101f57f0 con 0x7fdffc0047d0 2023-09-12T14:55:31.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.884+0000 7fe00cff9640 1 -- 172.21.15.131:0/4046481747 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe008012470 con 0x7fdffc0047d0 2023-09-12T14:55:31.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.884+0000 7fe0167a6640 1 -- 172.21.15.131:0/4046481747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0100acc10 con 0x7fdffc0047d0 2023-09-12T14:55:31.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.884+0000 7fe0167a6640 1 -- 172.21.15.131:0/4046481747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0101e55f0 con 0x7fdffc0047d0 2023-09-12T14:55:31.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.884+0000 7fe00cff9640 1 -- 172.21.15.131:0/4046481747 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fe0080125d0 con 0x7fdffc0047d0 2023-09-12T14:55:31.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.884+0000 7fe00cff9640 1 --2- 172.21.15.131:0/4046481747 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdff0065b50 0x7fdff0068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:31.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.888+0000 7fe00cff9640 1 -- 172.21.15.131:0/4046481747 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe008086920 con 0x7fdffc0047d0 2023-09-12T14:55:31.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.888+0000 7fe00f7fe640 1 --2- 172.21.15.131:0/4046481747 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdff0065b50 0x7fdff0068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:31.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.888+0000 7fe00f7fe640 1 --2- 172.21.15.131:0/4046481747 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdff0065b50 0x7fdff0068010 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdff80ae000 tx=0x7fdff8019fb0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:31.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:31.892+0000 7fe00cff9640 1 -- 172.21.15.131:0/4046481747 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fe0101e55f0 con 0x7fdffc0047d0 2023-09-12T14:55:32.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.044+0000 7fe0167a6640 1 -- 172.21.15.131:0/4046481747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe0101e55f0 con 0x7fdffc0047d0 2023-09-12T14:55:32.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.044+0000 7fe00cff9640 1 -- 172.21.15.131:0/4046481747 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe0101e55f0 con 0x7fdffc0047d0 2023-09-12T14:55:32.047 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-09-12T14:55:32.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.044+0000 7fe0167a6640 1 -- 172.21.15.131:0/4046481747 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdff0065b50 msgr2=0x7fdff0068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:32.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.044+0000 7fe0167a6640 1 --2- 172.21.15.131:0/4046481747 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdff0065b50 0x7fdff0068010 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdff80ae000 tx=0x7fdff8019fb0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.044+0000 7fe0167a6640 1 -- 172.21.15.131:0/4046481747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdffc0047d0 msgr2=0x7fe0101f4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:32.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.044+0000 7fe0167a6640 1 --2- 172.21.15.131:0/4046481747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdffc0047d0 0x7fe0101f4930 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe00800f630 tx=0x7fe00800faf0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.044+0000 7fe0167a6640 1 -- 172.21.15.131:0/4046481747 shutdown_connections 2023-09-12T14:55:32.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.044+0000 7fe0167a6640 1 --2- 172.21.15.131:0/4046481747 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdff0065b50 0x7fdff0068010 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.044+0000 7fe0167a6640 1 --2- 172.21.15.131:0/4046481747 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0101e1350 0x7fe0101e5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.044+0000 7fe0167a6640 1 --2- 172.21.15.131:0/4046481747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe01015aee0 0x7fe0100ac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.044+0000 7fe0167a6640 1 --2- 172.21.15.131:0/4046481747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdffc0047d0 0x7fe0101f4930 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.044+0000 7fe0167a6640 1 -- 172.21.15.131:0/4046481747 >> 172.21.15.131:0/4046481747 conn(0x7fe010069840 msgr2=0x7fe01006bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:32.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.044+0000 7fe0167a6640 1 -- 172.21.15.131:0/4046481747 shutdown_connections 2023-09-12T14:55:32.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.044+0000 7fe0167a6640 1 -- 172.21.15.131:0/4046481747 wait complete. 2023-09-12T14:55:32.067 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-12T14:55:32.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1f7c3640 1 -- 172.21.15.131:0/4208001270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c181574d0 msgr2=0x7f5c180cacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:32.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1f7c3640 1 --2- 172.21.15.131:0/4208001270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c181574d0 0x7f5c180cacd0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5c180cd990 tx=0x7f5c1001c870 comp rx=0 tx=0).stop 2023-09-12T14:55:32.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1f7c3640 1 -- 172.21.15.131:0/4208001270 shutdown_connections 2023-09-12T14:55:32.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1f7c3640 1 --2- 172.21.15.131:0/4208001270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c180c98d0 0x7f5c1806aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1f7c3640 1 --2- 172.21.15.131:0/4208001270 >> v2:172.21.15.187:3300/0 conn(0x7f5c180b2600 0x7f5c180c9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1f7c3640 1 --2- 172.21.15.131:0/4208001270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c181574d0 0x7f5c180cacd0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1f7c3640 1 -- 172.21.15.131:0/4208001270 >> 172.21.15.131:0/4208001270 conn(0x7f5c18069840 msgr2=0x7f5c180cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:32.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1f7c3640 1 -- 172.21.15.131:0/4208001270 shutdown_connections 2023-09-12T14:55:32.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1f7c3640 1 -- 172.21.15.131:0/4208001270 wait complete. 2023-09-12T14:55:32.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1f7c3640 1 Processor -- start 2023-09-12T14:55:32.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1f7c3640 1 -- start start 2023-09-12T14:55:32.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1f7c3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c180b2600 0x7f5c1815a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:32.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1f7c3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c180c98d0 0x7f5c181455f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:32.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1f7c3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c181574d0 0x7f5c18145d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:32.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1f7c3640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1815a0e0 con 0x7f5c180c98d0 2023-09-12T14:55:32.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1f7c3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c18145b30 con 0x7f5c181574d0 2023-09-12T14:55:32.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1f7c3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1814f870 con 0x7f5c180b2600 2023-09-12T14:55:32.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1cd5e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c180c98d0 0x7f5c181455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:32.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1cd5e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c180c98d0 0x7f5c181455f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59914/0 (socket says 172.21.15.131:59914) 2023-09-12T14:55:32.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1cd5e640 1 -- 172.21.15.131:0/795892725 learned_addr learned my addr 172.21.15.131:0/795892725 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:32.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1d55f640 1 --2- 172.21.15.131:0/795892725 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c180b2600 0x7f5c1815a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:32.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1dd60640 1 --2- 172.21.15.131:0/795892725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c181574d0 0x7f5c18145d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:32.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1cd5e640 1 -- 172.21.15.131:0/795892725 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c180b2600 msgr2=0x7f5c1815a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:32.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1cd5e640 1 --2- 172.21.15.131:0/795892725 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c180b2600 0x7f5c1815a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1cd5e640 1 -- 172.21.15.131:0/795892725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c181574d0 msgr2=0x7f5c18145d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:32.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1cd5e640 1 --2- 172.21.15.131:0/795892725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c181574d0 0x7f5c18145d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1cd5e640 1 -- 172.21.15.131:0/795892725 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c10009560 con 0x7f5c180c98d0 2023-09-12T14:55:32.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1d55f640 1 --2- 172.21.15.131:0/795892725 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c180b2600 0x7f5c1815a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:32.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1cd5e640 1 --2- 172.21.15.131:0/795892725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c180c98d0 0x7f5c181455f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00c970 tx=0x7f5c0c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:32.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c15ffb640 1 -- 172.21.15.131:0/795892725 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c007bb0 con 0x7f5c180c98d0 2023-09-12T14:55:32.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c15ffb640 1 -- 172.21.15.131:0/795892725 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5c0c007e00 con 0x7f5c180c98d0 2023-09-12T14:55:32.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1f7c3640 1 -- 172.21.15.131:0/795892725 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1814f590 con 0x7f5c180c98d0 2023-09-12T14:55:32.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c15ffb640 1 -- 172.21.15.131:0/795892725 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c005280 con 0x7f5c180c98d0 2023-09-12T14:55:32.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.192+0000 7f5c1f7c3640 1 -- 172.21.15.131:0/795892725 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c18159d70 con 0x7f5c180c98d0 2023-09-12T14:55:32.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.196+0000 7f5c1f7c3640 1 -- 172.21.15.131:0/795892725 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c1815a290 con 0x7f5c180c98d0 2023-09-12T14:55:32.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.196+0000 7f5c15ffb640 1 -- 172.21.15.131:0/795892725 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f5c0c0053e0 con 0x7f5c180c98d0 2023-09-12T14:55:32.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.196+0000 7f5c15ffb640 1 --2- 172.21.15.131:0/795892725 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5c04065b20 0x7f5c04067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:32.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.196+0000 7f5c15ffb640 1 -- 172.21.15.131:0/795892725 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c0c083980 con 0x7f5c180c98d0 2023-09-12T14:55:32.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.196+0000 7f5c1d55f640 1 --2- 172.21.15.131:0/795892725 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5c04065b20 0x7f5c04067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:32.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.196+0000 7f5c1d55f640 1 --2- 172.21.15.131:0/795892725 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5c04065b20 0x7f5c04067fe0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5c100af000 tx=0x7f5c1001b0b0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:32.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.200+0000 7f5c15ffb640 1 -- 172.21.15.131:0/795892725 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f5c1815a290 con 0x7f5c180c98d0 2023-09-12T14:55:32.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.368+0000 7f5c1f7c3640 1 -- 172.21.15.131:0/795892725 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5c18073580 con 0x7f5c180c98d0 2023-09-12T14:55:32.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.368+0000 7f5c15ffb640 1 -- 172.21.15.131:0/795892725 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5c0c01d070 con 0x7f5c180c98d0 2023-09-12T14:55:32.372 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2023-09-12T14:55:32.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.372+0000 7f5c1f7c3640 1 -- 172.21.15.131:0/795892725 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5c04065b20 msgr2=0x7f5c04067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:32.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.372+0000 7f5c1f7c3640 1 --2- 172.21.15.131:0/795892725 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5c04065b20 0x7f5c04067fe0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5c100af000 tx=0x7f5c1001b0b0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.372+0000 7f5c1f7c3640 1 -- 172.21.15.131:0/795892725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c180c98d0 msgr2=0x7f5c181455f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:32.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.372+0000 7f5c1f7c3640 1 --2- 172.21.15.131:0/795892725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c180c98d0 0x7f5c181455f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00c970 tx=0x7f5c0c00ce30 comp rx=0 tx=0).stop 2023-09-12T14:55:32.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.372+0000 7f5c1f7c3640 1 -- 172.21.15.131:0/795892725 shutdown_connections 2023-09-12T14:55:32.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.372+0000 7f5c1f7c3640 1 --2- 172.21.15.131:0/795892725 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5c04065b20 0x7f5c04067fe0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.372+0000 7f5c1f7c3640 1 --2- 172.21.15.131:0/795892725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c181574d0 0x7f5c18145d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.372+0000 7f5c1f7c3640 1 --2- 172.21.15.131:0/795892725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c180c98d0 0x7f5c181455f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.379 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.372+0000 7f5c1f7c3640 1 --2- 172.21.15.131:0/795892725 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c180b2600 0x7f5c1815a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.379 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.372+0000 7f5c1f7c3640 1 -- 172.21.15.131:0/795892725 >> 172.21.15.131:0/795892725 conn(0x7f5c18069840 msgr2=0x7f5c180ada70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:32.379 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.372+0000 7f5c1f7c3640 1 -- 172.21.15.131:0/795892725 shutdown_connections 2023-09-12T14:55:32.379 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.372+0000 7f5c1f7c3640 1 -- 172.21.15.131:0/795892725 wait complete. 2023-09-12T14:55:32.391 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:55:32.392 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-12T14:55:32.398 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:55:32.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbfb579640 1 -- 172.21.15.131:0/4014663940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbf41556e0 msgr2=0x7fcbf4158130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:32.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbfb579640 1 --2- 172.21.15.131:0/4014663940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbf41556e0 0x7fcbf4158130 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcbf40b05c0 tx=0x7fcbf001d2d0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbfb579640 1 -- 172.21.15.131:0/4014663940 shutdown_connections 2023-09-12T14:55:32.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbfb579640 1 --2- 172.21.15.131:0/4014663940 >> v2:172.21.15.187:3300/0 conn(0x7fcbf40b2100 0x7fcbf406be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbfb579640 1 --2- 172.21.15.131:0/4014663940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbf41556e0 0x7fcbf4158130 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbfb579640 1 -- 172.21.15.131:0/4014663940 >> 172.21.15.131:0/4014663940 conn(0x7fcbf406a950 msgr2=0x7fcbf40af5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:32.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbfb579640 1 -- 172.21.15.131:0/4014663940 shutdown_connections 2023-09-12T14:55:32.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbfb579640 1 -- 172.21.15.131:0/4014663940 wait complete. 2023-09-12T14:55:32.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbfb579640 1 Processor -- start 2023-09-12T14:55:32.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbfb579640 1 -- start start 2023-09-12T14:55:32.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbfb579640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbf406c3b0 0x7fcbf4149fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:32.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbfb579640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbf40b2100 0x7fcbf4159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:32.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbfb579640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbf41556e0 0x7fcbf415aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:32.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbfb579640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf414a500 con 0x7fcbf40b2100 2023-09-12T14:55:32.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbfb579640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf40cbf90 con 0x7fcbf41556e0 2023-09-12T14:55:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbfb579640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf4153cc0 con 0x7fcbf406c3b0 2023-09-12T14:55:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbf8b14640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbf40b2100 0x7fcbf4159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbf8b14640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbf40b2100 0x7fcbf4159ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59930/0 (socket says 172.21.15.131:59930) 2023-09-12T14:55:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbf8b14640 1 -- 172.21.15.131:0/713884389 learned_addr learned my addr 172.21.15.131:0/713884389 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbf9315640 1 --2- 172.21.15.131:0/713884389 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbf406c3b0 0x7fcbf4149fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:32.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbf9b16640 1 --2- 172.21.15.131:0/713884389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbf41556e0 0x7fcbf415aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:32.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbf8b14640 1 -- 172.21.15.131:0/713884389 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbf406c3b0 msgr2=0x7fcbf4149fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:32.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbf8b14640 1 --2- 172.21.15.131:0/713884389 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbf406c3b0 0x7fcbf4149fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbf8b14640 1 -- 172.21.15.131:0/713884389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbf41556e0 msgr2=0x7fcbf415aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:32.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbf8b14640 1 --2- 172.21.15.131:0/713884389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbf41556e0 0x7fcbf415aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbf8b14640 1 -- 172.21.15.131:0/713884389 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf0009560 con 0x7fcbf40b2100 2023-09-12T14:55:32.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbf8b14640 1 --2- 172.21.15.131:0/713884389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbf40b2100 0x7fcbf4159ba0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400c970 tx=0x7fcbe400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:32.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.564+0000 7fcbe9ffb640 1 -- 172.21.15.131:0/713884389 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4007bb0 con 0x7fcbf40b2100 2023-09-12T14:55:32.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.568+0000 7fcbe9ffb640 1 -- 172.21.15.131:0/713884389 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcbe4007e00 con 0x7fcbf40b2100 2023-09-12T14:55:32.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.568+0000 7fcbfb579640 1 -- 172.21.15.131:0/713884389 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf4153fa0 con 0x7fcbf40b2100 2023-09-12T14:55:32.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.568+0000 7fcbe9ffb640 1 -- 172.21.15.131:0/713884389 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4005280 con 0x7fcbf40b2100 2023-09-12T14:55:32.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.568+0000 7fcbfb579640 1 -- 172.21.15.131:0/713884389 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf4154560 con 0x7fcbf40b2100 2023-09-12T14:55:32.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.568+0000 7fcbfb579640 1 -- 172.21.15.131:0/713884389 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbf4073580 con 0x7fcbf40b2100 2023-09-12T14:55:32.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.568+0000 7fcbe9ffb640 1 -- 172.21.15.131:0/713884389 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fcbe40053e0 con 0x7fcbf40b2100 2023-09-12T14:55:32.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.568+0000 7fcbe9ffb640 1 --2- 172.21.15.131:0/713884389 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fcbd8065b20 0x7fcbd8067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:32.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.568+0000 7fcbe9ffb640 1 -- 172.21.15.131:0/713884389 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcbe40838a0 con 0x7fcbf40b2100 2023-09-12T14:55:32.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.568+0000 7fcbf9315640 1 --2- 172.21.15.131:0/713884389 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fcbd8065b20 0x7fcbd8067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:32.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.568+0000 7fcbf9315640 1 --2- 172.21.15.131:0/713884389 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fcbd8065b20 0x7fcbd8067fe0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00af000 tx=0x7fcbf001b0b0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:32.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.572+0000 7fcbe9ffb640 1 -- 172.21.15.131:0/713884389 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fcbe4051280 con 0x7fcbf40b2100 2023-09-12T14:55:32.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.728+0000 7fcbfb579640 1 -- 172.21.15.131:0/713884389 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbf40b1950 con 0x7fcbf40b2100 2023-09-12T14:55:32.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.732+0000 7fcbe9ffb640 1 -- 172.21.15.131:0/713884389 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fcbe401d070 con 0x7fcbf40b2100 2023-09-12T14:55:32.735 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:32.735 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:54:59.694778+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:54:55.689983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T14:54:57.029447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:55:32.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.732+0000 7fcbfb579640 1 -- 172.21.15.131:0/713884389 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fcbd8065b20 msgr2=0x7fcbd8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:32.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.732+0000 7fcbfb579640 1 --2- 172.21.15.131:0/713884389 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fcbd8065b20 0x7fcbd8067fe0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00af000 tx=0x7fcbf001b0b0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.732+0000 7fcbfb579640 1 -- 172.21.15.131:0/713884389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbf40b2100 msgr2=0x7fcbf4159ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:32.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.732+0000 7fcbfb579640 1 --2- 172.21.15.131:0/713884389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbf40b2100 0x7fcbf4159ba0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400c970 tx=0x7fcbe400ce30 comp rx=0 tx=0).stop 2023-09-12T14:55:32.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.732+0000 7fcbfb579640 1 -- 172.21.15.131:0/713884389 shutdown_connections 2023-09-12T14:55:32.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.732+0000 7fcbfb579640 1 --2- 172.21.15.131:0/713884389 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fcbd8065b20 0x7fcbd8067fe0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.732+0000 7fcbfb579640 1 --2- 172.21.15.131:0/713884389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbf41556e0 0x7fcbf415aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.732+0000 7fcbfb579640 1 --2- 172.21.15.131:0/713884389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbf40b2100 0x7fcbf4159ba0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.732+0000 7fcbfb579640 1 --2- 172.21.15.131:0/713884389 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbf406c3b0 0x7fcbf4149fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.732+0000 7fcbfb579640 1 -- 172.21.15.131:0/713884389 >> 172.21.15.131:0/713884389 conn(0x7fcbf406a950 msgr2=0x7fcbf415c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:32.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.732+0000 7fcbfb579640 1 -- 172.21.15.131:0/713884389 shutdown_connections 2023-09-12T14:55:32.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.732+0000 7fcbfb579640 1 -- 172.21.15.131:0/713884389 wait complete. 2023-09-12T14:55:32.754 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T14:54:45.654004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T14:54:55.689983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T14:54:57.029447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-12T14:55:32.755 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T14:55:32.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.880+0000 7f8a9fd2a640 1 -- 172.21.15.131:0/2587686263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a840047d0 msgr2=0x7f8a84004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:32.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.880+0000 7f8a9fd2a640 1 --2- 172.21.15.131:0/2587686263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a840047d0 0x7f8a84004c50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009b60 tx=0x7f8a8801cf90 comp rx=0 tx=0).stop 2023-09-12T14:55:32.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.880+0000 7f8a9e2c7640 1 -- 172.21.15.131:0/2587686263 reap_dead start 2023-09-12T14:55:32.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.880+0000 7f8a9fd2a640 1 -- 172.21.15.131:0/2587686263 shutdown_connections 2023-09-12T14:55:32.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.880+0000 7f8a9fd2a640 1 -- 172.21.15.131:0/2587686263 >> 172.21.15.131:0/2587686263 conn(0x7f8a98069840 msgr2=0x7f8a980cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:32.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.880+0000 7f8a9fd2a640 1 -- 172.21.15.131:0/2587686263 shutdown_connections 2023-09-12T14:55:32.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.880+0000 7f8a9fd2a640 1 -- 172.21.15.131:0/2587686263 wait complete. 2023-09-12T14:55:32.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.880+0000 7f8a9fd2a640 1 Processor -- start 2023-09-12T14:55:32.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.880+0000 7f8a9fd2a640 1 -- start start 2023-09-12T14:55:32.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.880+0000 7f8a9fd2a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a840047d0 0x7f8a981f5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:32.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.880+0000 7f8a9fd2a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a98142e20 0x7f8a981e2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:32.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.880+0000 7f8a9fd2a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a98152e70 0x7f8a981532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:32.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.880+0000 7f8a9fd2a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a98153830 con 0x7f8a840047d0 2023-09-12T14:55:32.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.880+0000 7f8a9fd2a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a981e61f0 con 0x7f8a98152e70 2023-09-12T14:55:32.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.880+0000 7f8a9fd2a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a981432b0 con 0x7f8a98142e20 2023-09-12T14:55:32.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.880+0000 7f8a9dac6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a840047d0 0x7f8a981f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:32.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.884+0000 7f8a9dac6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a840047d0 0x7f8a981f5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59936/0 (socket says 172.21.15.131:59936) 2023-09-12T14:55:32.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.884+0000 7f8a9dac6640 1 -- 172.21.15.131:0/3454961961 learned_addr learned my addr 172.21.15.131:0/3454961961 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:32.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.884+0000 7f8a9e2c7640 1 --2- 172.21.15.131:0/3454961961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a98152e70 0x7f8a981532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:32.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.884+0000 7f8a9d2c5640 1 --2- 172.21.15.131:0/3454961961 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a98142e20 0x7f8a981e2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:32.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.884+0000 7f8a9dac6640 1 -- 172.21.15.131:0/3454961961 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a98142e20 msgr2=0x7f8a981e2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:32.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.884+0000 7f8a9dac6640 1 --2- 172.21.15.131:0/3454961961 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a98142e20 0x7f8a981e2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.884+0000 7f8a9dac6640 1 -- 172.21.15.131:0/3454961961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a98152e70 msgr2=0x7f8a981532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:32.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.884+0000 7f8a9dac6640 1 --2- 172.21.15.131:0/3454961961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a98152e70 0x7f8a981532f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:32.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.884+0000 7f8a9dac6640 1 -- 172.21.15.131:0/3454961961 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a98143590 con 0x7f8a840047d0 2023-09-12T14:55:32.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.884+0000 7f8a9d2c5640 1 --2- 172.21.15.131:0/3454961961 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a98142e20 0x7f8a981e2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:32.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.884+0000 7f8a9e2c7640 1 --2- 172.21.15.131:0/3454961961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a98152e70 0x7f8a981532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:32.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.884+0000 7f8a9dac6640 1 --2- 172.21.15.131:0/3454961961 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a840047d0 0x7f8a981f5250 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8a94006fd0 tx=0x7f8a94001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:32.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.884+0000 7f8a8e7fc640 1 -- 172.21.15.131:0/3454961961 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94011870 con 0x7f8a840047d0 2023-09-12T14:55:32.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.884+0000 7f8a8e7fc640 1 -- 172.21.15.131:0/3454961961 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8a94011ac0 con 0x7f8a840047d0 2023-09-12T14:55:32.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.884+0000 7f8a9fd2a640 1 -- 172.21.15.131:0/3454961961 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a98143870 con 0x7f8a840047d0 2023-09-12T14:55:32.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.884+0000 7f8a8e7fc640 1 -- 172.21.15.131:0/3454961961 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94001d50 con 0x7f8a840047d0 2023-09-12T14:55:32.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.884+0000 7f8a9fd2a640 1 -- 172.21.15.131:0/3454961961 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a98143e30 con 0x7f8a840047d0 2023-09-12T14:55:32.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.884+0000 7f8a9fd2a640 1 -- 172.21.15.131:0/3454961961 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a98073580 con 0x7f8a840047d0 2023-09-12T14:55:32.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.884+0000 7f8a8e7fc640 1 -- 172.21.15.131:0/3454961961 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f8a9401a400 con 0x7f8a840047d0 2023-09-12T14:55:32.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.884+0000 7f8a8e7fc640 1 --2- 172.21.15.131:0/3454961961 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8a7c065b20 0x7f8a7c067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:32.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.884+0000 7f8a8e7fc640 1 -- 172.21.15.131:0/3454961961 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a940840f0 con 0x7f8a840047d0 2023-09-12T14:55:32.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.884+0000 7f8a9d2c5640 1 --2- 172.21.15.131:0/3454961961 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8a7c065b20 0x7f8a7c067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:32.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.884+0000 7f8a9d2c5640 1 --2- 172.21.15.131:0/3454961961 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8a7c065b20 0x7f8a7c067fe0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8a880ae000 tx=0x7f8a88019d30 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:32.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:32.888+0000 7f8a8e7fc640 1 -- 172.21.15.131:0/3454961961 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f8a94051a40 con 0x7f8a840047d0 2023-09-12T14:55:33.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.040+0000 7f8a9fd2a640 1 -- 172.21.15.131:0/3454961961 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8a98154250 con 0x7f8a840047d0 2023-09-12T14:55:33.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.044+0000 7f8a8e7fc640 1 -- 172.21.15.131:0/3454961961 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8a94055060 con 0x7f8a840047d0 2023-09-12T14:55:33.046 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-09-12T14:55:33.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.044+0000 7f8a9fd2a640 1 -- 172.21.15.131:0/3454961961 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8a7c065b20 msgr2=0x7f8a7c067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:33.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.044+0000 7f8a9fd2a640 1 --2- 172.21.15.131:0/3454961961 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8a7c065b20 0x7f8a7c067fe0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8a880ae000 tx=0x7f8a88019d30 comp rx=0 tx=0).stop 2023-09-12T14:55:33.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.044+0000 7f8a9fd2a640 1 -- 172.21.15.131:0/3454961961 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a840047d0 msgr2=0x7f8a981f5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:33.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.044+0000 7f8a9fd2a640 1 --2- 172.21.15.131:0/3454961961 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a840047d0 0x7f8a981f5250 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8a94006fd0 tx=0x7f8a94001640 comp rx=0 tx=0).stop 2023-09-12T14:55:33.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.044+0000 7f8a9fd2a640 1 -- 172.21.15.131:0/3454961961 shutdown_connections 2023-09-12T14:55:33.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.044+0000 7f8a9fd2a640 1 --2- 172.21.15.131:0/3454961961 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8a7c065b20 0x7f8a7c067fe0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.044+0000 7f8a9fd2a640 1 --2- 172.21.15.131:0/3454961961 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a98142e20 0x7f8a981e2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.044+0000 7f8a9fd2a640 1 --2- 172.21.15.131:0/3454961961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a98152e70 0x7f8a981532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.044+0000 7f8a9fd2a640 1 --2- 172.21.15.131:0/3454961961 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a840047d0 0x7f8a981f5250 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.044+0000 7f8a9fd2a640 1 -- 172.21.15.131:0/3454961961 >> 172.21.15.131:0/3454961961 conn(0x7f8a98069840 msgr2=0x7f8a980aa8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:33.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.044+0000 7f8a9fd2a640 1 -- 172.21.15.131:0/3454961961 shutdown_connections 2023-09-12T14:55:33.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.044+0000 7f8a9fd2a640 1 -- 172.21.15.131:0/3454961961 wait complete. 2023-09-12T14:55:33.065 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T14:55:33.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.188+0000 7fa41f39e640 1 -- 172.21.15.131:0/2628064765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4180cc430 msgr2=0x7fa4180c9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:33.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.188+0000 7fa41f39e640 1 --2- 172.21.15.131:0/2628064765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4180cc430 0x7fa4180c9850 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa418071020 tx=0x7fa41401c810 comp rx=0 tx=0).stop 2023-09-12T14:55:33.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.188+0000 7fa41f39e640 1 -- 172.21.15.131:0/2628064765 shutdown_connections 2023-09-12T14:55:33.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.188+0000 7fa41f39e640 1 --2- 172.21.15.131:0/2628064765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4180cc430 0x7fa4180c9850 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.188+0000 7fa41f39e640 1 --2- 172.21.15.131:0/2628064765 >> v2:172.21.15.187:3300/0 conn(0x7fa4180abae0 0x7fa4180c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.188+0000 7fa41f39e640 1 --2- 172.21.15.131:0/2628064765 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa418158d70 0x7fa41806b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.188+0000 7fa41f39e640 1 -- 172.21.15.131:0/2628064765 >> 172.21.15.131:0/2628064765 conn(0x7fa418069840 msgr2=0x7fa4180cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:33.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.188+0000 7fa41f39e640 1 -- 172.21.15.131:0/2628064765 shutdown_connections 2023-09-12T14:55:33.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.188+0000 7fa41f39e640 1 -- 172.21.15.131:0/2628064765 wait complete. 2023-09-12T14:55:33.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.192+0000 7fa41f39e640 1 Processor -- start 2023-09-12T14:55:33.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.192+0000 7fa41f39e640 1 -- start start 2023-09-12T14:55:33.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.192+0000 7fa41f39e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4180abae0 0x7fa41814eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:33.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.192+0000 7fa41f39e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4180cc430 0x7fa41814e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:33.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.192+0000 7fa41f39e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa418158d70 0x7fa4180aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:33.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.192+0000 7fa41f39e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa418159fb0 con 0x7fa4180cc430 2023-09-12T14:55:33.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.192+0000 7fa41f39e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa418159610 con 0x7fa4180abae0 2023-09-12T14:55:33.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.192+0000 7fa41f39e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4181598f0 con 0x7fa418158d70 2023-09-12T14:55:33.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.192+0000 7fa41c939640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4180cc430 0x7fa41814e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:33.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.192+0000 7fa41c939640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4180cc430 0x7fa41814e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59958/0 (socket says 172.21.15.131:59958) 2023-09-12T14:55:33.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.192+0000 7fa41c939640 1 -- 172.21.15.131:0/3092323670 learned_addr learned my addr 172.21.15.131:0/3092323670 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:33.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.192+0000 7fa41d13a640 1 --2- 172.21.15.131:0/3092323670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4180abae0 0x7fa41814eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:33.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.192+0000 7fa41c939640 1 -- 172.21.15.131:0/3092323670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa418158d70 msgr2=0x7fa4180aa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:33.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.192+0000 7fa41c939640 1 --2- 172.21.15.131:0/3092323670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa418158d70 0x7fa4180aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.192+0000 7fa41c939640 1 -- 172.21.15.131:0/3092323670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4180abae0 msgr2=0x7fa41814eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:33.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.192+0000 7fa41c939640 1 --2- 172.21.15.131:0/3092323670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4180abae0 0x7fa41814eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.192+0000 7fa41c939640 1 -- 172.21.15.131:0/3092323670 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa414009560 con 0x7fa4180cc430 2023-09-12T14:55:33.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.192+0000 7fa41c939640 1 --2- 172.21.15.131:0/3092323670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4180cc430 0x7fa41814e890 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa40800c970 tx=0x7fa40800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:33.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.192+0000 7fa40dffb640 1 -- 172.21.15.131:0/3092323670 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa408007bb0 con 0x7fa4180cc430 2023-09-12T14:55:33.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.192+0000 7fa40dffb640 1 -- 172.21.15.131:0/3092323670 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa408007e00 con 0x7fa4180cc430 2023-09-12T14:55:33.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.192+0000 7fa40dffb640 1 -- 172.21.15.131:0/3092323670 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40800f6b0 con 0x7fa4180cc430 2023-09-12T14:55:33.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.192+0000 7fa41f39e640 1 -- 172.21.15.131:0/3092323670 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa418159bd0 con 0x7fa4180cc430 2023-09-12T14:55:33.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.192+0000 7fa41f39e640 1 -- 172.21.15.131:0/3092323670 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4180ae270 con 0x7fa4180cc430 2023-09-12T14:55:33.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.192+0000 7fa41f39e640 1 -- 172.21.15.131:0/3092323670 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa418073580 con 0x7fa4180cc430 2023-09-12T14:55:33.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.196+0000 7fa40dffb640 1 -- 172.21.15.131:0/3092323670 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fa40800f810 con 0x7fa4180cc430 2023-09-12T14:55:33.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.196+0000 7fa40dffb640 1 --2- 172.21.15.131:0/3092323670 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa3fc065b50 0x7fa3fc068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:33.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.196+0000 7fa40dffb640 1 -- 172.21.15.131:0/3092323670 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa408083a50 con 0x7fa4180cc430 2023-09-12T14:55:33.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.196+0000 7fa41d13a640 1 --2- 172.21.15.131:0/3092323670 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa3fc065b50 0x7fa3fc068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:33.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.196+0000 7fa41d13a640 1 --2- 172.21.15.131:0/3092323670 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa3fc065b50 0x7fa3fc068010 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa410005c50 tx=0x7fa41000a3b0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:33.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.200+0000 7fa40dffb640 1 -- 172.21.15.131:0/3092323670 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fa4080513a0 con 0x7fa4180cc430 2023-09-12T14:55:33.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.340+0000 7fa41f39e640 1 -- 172.21.15.131:0/3092323670 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa4180cd4d0 con 0x7fa4180cc430 2023-09-12T14:55:33.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.340+0000 7fa40dffb640 1 -- 172.21.15.131:0/3092323670 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa40801d020 con 0x7fa4180cc430 2023-09-12T14:55:33.344 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-09-12T14:55:33.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.344+0000 7fa41f39e640 1 -- 172.21.15.131:0/3092323670 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa3fc065b50 msgr2=0x7fa3fc068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:33.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.344+0000 7fa41f39e640 1 --2- 172.21.15.131:0/3092323670 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa3fc065b50 0x7fa3fc068010 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa410005c50 tx=0x7fa41000a3b0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.344+0000 7fa41f39e640 1 -- 172.21.15.131:0/3092323670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4180cc430 msgr2=0x7fa41814e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:33.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.344+0000 7fa41f39e640 1 --2- 172.21.15.131:0/3092323670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4180cc430 0x7fa41814e890 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa40800c970 tx=0x7fa40800ce30 comp rx=0 tx=0).stop 2023-09-12T14:55:33.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.344+0000 7fa41f39e640 1 -- 172.21.15.131:0/3092323670 shutdown_connections 2023-09-12T14:55:33.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.344+0000 7fa41f39e640 1 --2- 172.21.15.131:0/3092323670 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa3fc065b50 0x7fa3fc068010 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.344+0000 7fa41f39e640 1 --2- 172.21.15.131:0/3092323670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa418158d70 0x7fa4180aa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.344+0000 7fa41f39e640 1 --2- 172.21.15.131:0/3092323670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4180cc430 0x7fa41814e890 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.344+0000 7fa41f39e640 1 --2- 172.21.15.131:0/3092323670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4180abae0 0x7fa41814eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.344+0000 7fa41f39e640 1 -- 172.21.15.131:0/3092323670 >> 172.21.15.131:0/3092323670 conn(0x7fa418069840 msgr2=0x7fa4180ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:33.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.344+0000 7fa41f39e640 1 -- 172.21.15.131:0/3092323670 shutdown_connections 2023-09-12T14:55:33.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.344+0000 7fa41f39e640 1 -- 172.21.15.131:0/3092323670 wait complete. 2023-09-12T14:55:33.363 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T14:55:33.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd7828640 1 -- 172.21.15.131:0/3925277119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bd00abb40 msgr2=0x7f4bd006b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:33.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd7828640 1 --2- 172.21.15.131:0/3925277119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bd00abb40 0x7f4bd006b4a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0009840 tx=0x7f4bc001c7d0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd7828640 1 -- 172.21.15.131:0/3925277119 shutdown_connections 2023-09-12T14:55:33.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd7828640 1 --2- 172.21.15.131:0/3925277119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bd00abb40 0x7f4bd006b4a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd7828640 1 --2- 172.21.15.131:0/3925277119 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bd0156930 0x7f4bd0159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd7828640 1 -- 172.21.15.131:0/3925277119 >> 172.21.15.131:0/3925277119 conn(0x7f4bd0069840 msgr2=0x7f4bd00cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:33.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd7828640 1 -- 172.21.15.131:0/3925277119 shutdown_connections 2023-09-12T14:55:33.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd7828640 1 -- 172.21.15.131:0/3925277119 wait complete. 2023-09-12T14:55:33.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd7828640 1 Processor -- start 2023-09-12T14:55:33.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd7828640 1 -- start start 2023-09-12T14:55:33.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd7828640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4bd00abb40 0x7f4bd0144ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:33.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd7828640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bd00c9370 0x7f4bd014f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:33.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd7828640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bd0156930 0x7f4bd014c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:33.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd7828640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd0070ca0 con 0x7f4bd0156930 2023-09-12T14:55:33.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd7828640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd014f9c0 con 0x7f4bd00abb40 2023-09-12T14:55:33.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd7828640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd014fca0 con 0x7f4bd00c9370 2023-09-12T14:55:33.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd5dc5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bd0156930 0x7f4bd014c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:33.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd5dc5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bd0156930 0x7f4bd014c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59982/0 (socket says 172.21.15.131:59982) 2023-09-12T14:55:33.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd5dc5640 1 -- 172.21.15.131:0/4117906254 learned_addr learned my addr 172.21.15.131:0/4117906254 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:33.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd4dc3640 1 --2- 172.21.15.131:0/4117906254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bd00c9370 0x7f4bd014f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:33.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd55c4640 1 --2- 172.21.15.131:0/4117906254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4bd00abb40 0x7f4bd0144ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:33.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd5dc5640 1 -- 172.21.15.131:0/4117906254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bd00c9370 msgr2=0x7f4bd014f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:33.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd5dc5640 1 --2- 172.21.15.131:0/4117906254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bd00c9370 0x7f4bd014f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd5dc5640 1 -- 172.21.15.131:0/4117906254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4bd00abb40 msgr2=0x7f4bd0144ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:33.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd5dc5640 1 --2- 172.21.15.131:0/4117906254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4bd00abb40 0x7f4bd0144ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd5dc5640 1 -- 172.21.15.131:0/4117906254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc00094a0 con 0x7f4bd0156930 2023-09-12T14:55:33.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd55c4640 1 --2- 172.21.15.131:0/4117906254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4bd00abb40 0x7f4bd0144ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:33.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd5dc5640 1 --2- 172.21.15.131:0/4117906254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bd0156930 0x7f4bd014c3d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800cb40 tx=0x7f4bc800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:33.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bc5ffb640 1 -- 172.21.15.131:0/4117906254 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc800aa50 con 0x7f4bd0156930 2023-09-12T14:55:33.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bc5ffb640 1 -- 172.21.15.131:0/4117906254 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4bc800aca0 con 0x7f4bd0156930 2023-09-12T14:55:33.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd7828640 1 -- 172.21.15.131:0/4117906254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd014ff80 con 0x7f4bd0156930 2023-09-12T14:55:33.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bc5ffb640 1 -- 172.21.15.131:0/4117906254 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8012680 con 0x7f4bd0156930 2023-09-12T14:55:33.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.480+0000 7f4bd7828640 1 -- 172.21.15.131:0/4117906254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd0150540 con 0x7f4bd0156930 2023-09-12T14:55:33.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.484+0000 7f4bd7828640 1 -- 172.21.15.131:0/4117906254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd0150760 con 0x7f4bd0156930 2023-09-12T14:55:33.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.484+0000 7f4bc5ffb640 1 -- 172.21.15.131:0/4117906254 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f4bc801a3b0 con 0x7f4bd0156930 2023-09-12T14:55:33.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.484+0000 7f4bc5ffb640 1 --2- 172.21.15.131:0/4117906254 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4bb4065b20 0x7f4bb4067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:33.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.484+0000 7f4bc5ffb640 1 -- 172.21.15.131:0/4117906254 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4bc80836f0 con 0x7f4bd0156930 2023-09-12T14:55:33.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.484+0000 7f4bd55c4640 1 --2- 172.21.15.131:0/4117906254 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4bb4065b20 0x7f4bb4067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:33.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.484+0000 7f4bd55c4640 1 --2- 172.21.15.131:0/4117906254 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4bb4065b20 0x7f4bb4067fe0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc005fd0 tx=0x7f4bcc005de0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:33.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.488+0000 7f4bc5ffb640 1 -- 172.21.15.131:0/4117906254 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f4bc8051040 con 0x7f4bd0156930 2023-09-12T14:55:33.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.628+0000 7f4bd7828640 1 -- 172.21.15.131:0/4117906254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4bd0073580 con 0x7f4bd0156930 2023-09-12T14:55:33.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.628+0000 7f4bc5ffb640 1 -- 172.21.15.131:0/4117906254 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4bc8054660 con 0x7f4bd0156930 2023-09-12T14:55:33.633 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-09-12T14:55:33.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.632+0000 7f4bd7828640 1 -- 172.21.15.131:0/4117906254 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4bb4065b20 msgr2=0x7f4bb4067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:33.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.632+0000 7f4bd7828640 1 --2- 172.21.15.131:0/4117906254 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4bb4065b20 0x7f4bb4067fe0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc005fd0 tx=0x7f4bcc005de0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.632+0000 7f4bd7828640 1 -- 172.21.15.131:0/4117906254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bd0156930 msgr2=0x7f4bd014c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:33.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.632+0000 7f4bd7828640 1 --2- 172.21.15.131:0/4117906254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bd0156930 0x7f4bd014c3d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800cb40 tx=0x7f4bc800cc30 comp rx=0 tx=0).stop 2023-09-12T14:55:33.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.632+0000 7f4bd7828640 1 -- 172.21.15.131:0/4117906254 shutdown_connections 2023-09-12T14:55:33.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.632+0000 7f4bd7828640 1 --2- 172.21.15.131:0/4117906254 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4bb4065b20 0x7f4bb4067fe0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.632+0000 7f4bd7828640 1 --2- 172.21.15.131:0/4117906254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bd00c9370 0x7f4bd014f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.632+0000 7f4bd7828640 1 --2- 172.21.15.131:0/4117906254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bd0156930 0x7f4bd014c3d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.632+0000 7f4bd7828640 1 --2- 172.21.15.131:0/4117906254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4bd00abb40 0x7f4bd0144ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.632+0000 7f4bd7828640 1 -- 172.21.15.131:0/4117906254 >> 172.21.15.131:0/4117906254 conn(0x7f4bd0069840 msgr2=0x7f4bd0157a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:33.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.632+0000 7f4bd7828640 1 -- 172.21.15.131:0/4117906254 shutdown_connections 2023-09-12T14:55:33.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.632+0000 7f4bd7828640 1 -- 172.21.15.131:0/4117906254 wait complete. 2023-09-12T14:55:33.651 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-12T14:55:33.776 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.772+0000 7fb5e0e4d640 1 -- 172.21.15.131:0/2303606360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5c40052b0 msgr2=0x7fb5c400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:33.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.772+0000 7fb5e0e4d640 1 --2- 172.21.15.131:0/2303606360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5c40052b0 0x7fb5c400b990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4001d70 tx=0x7fb5d401cae0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.772+0000 7fb5e0e4d640 1 -- 172.21.15.131:0/2303606360 shutdown_connections 2023-09-12T14:55:33.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.772+0000 7fb5e0e4d640 1 --2- 172.21.15.131:0/2303606360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5c40052b0 0x7fb5c400b990 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.772+0000 7fb5e0e4d640 1 -- 172.21.15.131:0/2303606360 >> 172.21.15.131:0/2303606360 conn(0x7fb5dc0ceae0 msgr2=0x7fb5dc0ceef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:33.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.772+0000 7fb5e0e4d640 1 -- 172.21.15.131:0/2303606360 shutdown_connections 2023-09-12T14:55:33.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.772+0000 7fb5e0e4d640 1 -- 172.21.15.131:0/2303606360 wait complete. 2023-09-12T14:55:33.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.772+0000 7fb5e0e4d640 1 Processor -- start 2023-09-12T14:55:33.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.776+0000 7fb5e0e4d640 1 -- start start 2023-09-12T14:55:33.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.776+0000 7fb5e0e4d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5c40052b0 0x7fb5dc152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:33.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.776+0000 7fb5e0e4d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5dc0c33d0 0x7fb5dc0ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:33.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.776+0000 7fb5e0e4d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5dc0ad540 0x7fb5dc0c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:33.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.776+0000 7fb5e0e4d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc0c62a0 con 0x7fb5c40052b0 2023-09-12T14:55:33.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.776+0000 7fb5e0e4d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc0ad9c0 con 0x7fb5dc0ad540 2023-09-12T14:55:33.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.776+0000 7fb5e0e4d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc0c81d0 con 0x7fb5dc0c33d0 2023-09-12T14:55:33.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.776+0000 7fb5da59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5c40052b0 0x7fb5dc152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:33.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.776+0000 7fb5da59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5c40052b0 0x7fb5dc152fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59990/0 (socket says 172.21.15.131:59990) 2023-09-12T14:55:33.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.776+0000 7fb5da59c640 1 -- 172.21.15.131:0/745428207 learned_addr learned my addr 172.21.15.131:0/745428207 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:33.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.776+0000 7fb5dad9d640 1 --2- 172.21.15.131:0/745428207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5dc0ad540 0x7fb5dc0c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:33.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.776+0000 7fb5d9d9b640 1 --2- 172.21.15.131:0/745428207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5dc0c33d0 0x7fb5dc0ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.776+0000 7fb5da59c640 1 -- 172.21.15.131:0/745428207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5dc0c33d0 msgr2=0x7fb5dc0ad000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.776+0000 7fb5da59c640 1 --2- 172.21.15.131:0/745428207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5dc0c33d0 0x7fb5dc0ad000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.776+0000 7fb5da59c640 1 -- 172.21.15.131:0/745428207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5dc0ad540 msgr2=0x7fb5dc0c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.776+0000 7fb5da59c640 1 --2- 172.21.15.131:0/745428207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5dc0ad540 0x7fb5dc0c7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.776+0000 7fb5da59c640 1 -- 172.21.15.131:0/745428207 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5dc0c5c50 con 0x7fb5c40052b0 2023-09-12T14:55:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.776+0000 7fb5da59c640 1 --2- 172.21.15.131:0/745428207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5c40052b0 0x7fb5dc152fb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc000fc0 tx=0x7fb5cc00fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:33.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.776+0000 7fb5caffd640 1 -- 172.21.15.131:0/745428207 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc0118a0 con 0x7fb5c40052b0 2023-09-12T14:55:33.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.776+0000 7fb5caffd640 1 -- 172.21.15.131:0/745428207 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb5cc011af0 con 0x7fb5c40052b0 2023-09-12T14:55:33.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.776+0000 7fb5e0e4d640 1 -- 172.21.15.131:0/745428207 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5dc0c5f30 con 0x7fb5c40052b0 2023-09-12T14:55:33.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.776+0000 7fb5caffd640 1 -- 172.21.15.131:0/745428207 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc011ea0 con 0x7fb5c40052b0 2023-09-12T14:55:33.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.776+0000 7fb5e0e4d640 1 -- 172.21.15.131:0/745428207 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5dc0c84b0 con 0x7fb5c40052b0 2023-09-12T14:55:33.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.776+0000 7fb5e0e4d640 1 -- 172.21.15.131:0/745428207 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5dc073580 con 0x7fb5c40052b0 2023-09-12T14:55:33.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.776+0000 7fb5caffd640 1 -- 172.21.15.131:0/745428207 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fb5cc012470 con 0x7fb5c40052b0 2023-09-12T14:55:33.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.776+0000 7fb5caffd640 1 --2- 172.21.15.131:0/745428207 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb5bc065b50 0x7fb5bc068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:33.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.776+0000 7fb5caffd640 1 -- 172.21.15.131:0/745428207 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5cc086da0 con 0x7fb5c40052b0 2023-09-12T14:55:33.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.780+0000 7fb5d9d9b640 1 --2- 172.21.15.131:0/745428207 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb5bc065b50 0x7fb5bc068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:33.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.780+0000 7fb5d9d9b640 1 --2- 172.21.15.131:0/745428207 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb5bc065b50 0x7fb5bc068010 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc0c5240 tx=0x7fb5d000c500 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:33.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.780+0000 7fb5caffd640 1 -- 172.21.15.131:0/745428207 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fb5cc054760 con 0x7fb5c40052b0 2023-09-12T14:55:33.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.944+0000 7fb5e0e4d640 1 -- 172.21.15.131:0/745428207 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb5dc0c5870 con 0x7fb5c40052b0 2023-09-12T14:55:33.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.944+0000 7fb5caffd640 1 -- 172.21.15.131:0/745428207 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb5cc057d80 con 0x7fb5c40052b0 2023-09-12T14:55:33.949 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2023-09-12T14:55:33.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.948+0000 7fb5e0e4d640 1 -- 172.21.15.131:0/745428207 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb5bc065b50 msgr2=0x7fb5bc068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:33.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.948+0000 7fb5e0e4d640 1 --2- 172.21.15.131:0/745428207 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb5bc065b50 0x7fb5bc068010 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc0c5240 tx=0x7fb5d000c500 comp rx=0 tx=0).stop 2023-09-12T14:55:33.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.948+0000 7fb5e0e4d640 1 -- 172.21.15.131:0/745428207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5c40052b0 msgr2=0x7fb5dc152fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:33.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.948+0000 7fb5e0e4d640 1 --2- 172.21.15.131:0/745428207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5c40052b0 0x7fb5dc152fb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc000fc0 tx=0x7fb5cc00fa30 comp rx=0 tx=0).stop 2023-09-12T14:55:33.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.948+0000 7fb5e0e4d640 1 -- 172.21.15.131:0/745428207 shutdown_connections 2023-09-12T14:55:33.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.948+0000 7fb5e0e4d640 1 --2- 172.21.15.131:0/745428207 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb5bc065b50 0x7fb5bc068010 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.948+0000 7fb5e0e4d640 1 --2- 172.21.15.131:0/745428207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5dc0c33d0 0x7fb5dc0ad000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.948+0000 7fb5e0e4d640 1 --2- 172.21.15.131:0/745428207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5dc0ad540 0x7fb5dc0c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.948+0000 7fb5e0e4d640 1 --2- 172.21.15.131:0/745428207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5c40052b0 0x7fb5dc152fb0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:33.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.948+0000 7fb5e0e4d640 1 -- 172.21.15.131:0/745428207 >> 172.21.15.131:0/745428207 conn(0x7fb5dc0ceae0 msgr2=0x7fb5dc0aef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:33.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.948+0000 7fb5e0e4d640 1 -- 172.21.15.131:0/745428207 shutdown_connections 2023-09-12T14:55:33.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:33.948+0000 7fb5e0e4d640 1 -- 172.21.15.131:0/745428207 wait complete. 2023-09-12T14:55:33.967 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi131.front.sepia.ceph.com... 2023-09-12T14:55:33.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:33.968 DEBUG:teuthology.orchestra.run.smithi131:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-12T14:55:34.067 INFO:teuthology.orchestra.run:waiting for 300 2023-09-12T14:55:34.067 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2023-09-12T14:55:34.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:34.068 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-12T14:55:34.079 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:55:34.079 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-12T14:55:34.086 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:55:34.086 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-12T14:55:34.132 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:55:34.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.264+0000 7fe8d9b1c640 1 -- 172.21.15.131:0/1313987513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8c00039b0 msgr2=0x7fe8c0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:34.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.264+0000 7fe8d9b1c640 1 --2- 172.21.15.131:0/1313987513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8c00039b0 0x7fe8c0003dd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400bb10 tx=0x7fe8c401cd50 comp rx=0 tx=0).stop 2023-09-12T14:55:34.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.264+0000 7fe8d3fff640 1 -- 172.21.15.131:0/1313987513 reap_dead start 2023-09-12T14:55:34.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.264+0000 7fe8d9b1c640 1 -- 172.21.15.131:0/1313987513 shutdown_connections 2023-09-12T14:55:34.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.264+0000 7fe8d9b1c640 1 -- 172.21.15.131:0/1313987513 >> 172.21.15.131:0/1313987513 conn(0x7fe8d409a200 msgr2=0x7fe8d40d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:34.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.264+0000 7fe8d9b1c640 1 -- 172.21.15.131:0/1313987513 shutdown_connections 2023-09-12T14:55:34.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.264+0000 7fe8d9b1c640 1 -- 172.21.15.131:0/1313987513 wait complete. 2023-09-12T14:55:34.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.264+0000 7fe8d9b1c640 1 Processor -- start 2023-09-12T14:55:34.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.264+0000 7fe8d9b1c640 1 -- start start 2023-09-12T14:55:34.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d9b1c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8c00039b0 0x7fe8d40c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:34.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d9b1c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8d40c94a0 0x7fe8d40c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:34.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d9b1c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8d40c4ee0 0x7fe8d415a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:34.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d9b1c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d41598e0 con 0x7fe8d40c4ee0 2023-09-12T14:55:34.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d9b1c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d4159bc0 con 0x7fe8c00039b0 2023-09-12T14:55:34.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d9b1c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d4159ea0 con 0x7fe8d40c94a0 2023-09-12T14:55:34.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8d40c4ee0 0x7fe8d415a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:34.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8d40c4ee0 0x7fe8d415a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60004/0 (socket says 172.21.15.131:60004) 2023-09-12T14:55:34.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d3fff640 1 -- 172.21.15.131:0/1308686777 learned_addr learned my addr 172.21.15.131:0/1308686777 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:34.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d37fe640 1 --2- 172.21.15.131:0/1308686777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8c00039b0 0x7fe8d40c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:34.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8cbfff640 1 --2- 172.21.15.131:0/1308686777 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8d40c94a0 0x7fe8d40c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:34.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d3fff640 1 -- 172.21.15.131:0/1308686777 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8d40c94a0 msgr2=0x7fe8d40c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:34.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d3fff640 1 --2- 172.21.15.131:0/1308686777 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8d40c94a0 0x7fe8d40c49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:34.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d3fff640 1 -- 172.21.15.131:0/1308686777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8c00039b0 msgr2=0x7fe8d40c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:34.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d3fff640 1 --2- 172.21.15.131:0/1308686777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8c00039b0 0x7fe8d40c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:34.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d3fff640 1 -- 172.21.15.131:0/1308686777 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d415a180 con 0x7fe8d40c4ee0 2023-09-12T14:55:34.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d37fe640 1 --2- 172.21.15.131:0/1308686777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8c00039b0 0x7fe8d40c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:55:34.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d3fff640 1 --2- 172.21.15.131:0/1308686777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8d40c4ee0 0x7fe8d415a980 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4089db0 tx=0x7fe8cc0019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:34.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d0ff9640 1 -- 172.21.15.131:0/1308686777 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc011870 con 0x7fe8d40c4ee0 2023-09-12T14:55:34.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d0ff9640 1 -- 172.21.15.131:0/1308686777 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe8cc011ae0 con 0x7fe8d40c4ee0 2023-09-12T14:55:34.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d9b1c640 1 -- 172.21.15.131:0/1308686777 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d415a460 con 0x7fe8d40c4ee0 2023-09-12T14:55:34.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d0ff9640 1 -- 172.21.15.131:0/1308686777 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc0020e0 con 0x7fe8d40c4ee0 2023-09-12T14:55:34.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d9b1c640 1 -- 172.21.15.131:0/1308686777 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d415aec0 con 0x7fe8d40c4ee0 2023-09-12T14:55:34.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d9b1c640 1 -- 172.21.15.131:0/1308686777 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d4073580 con 0x7fe8d40c4ee0 2023-09-12T14:55:34.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d0ff9640 1 -- 172.21.15.131:0/1308686777 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fe8cc01a400 con 0x7fe8d40c4ee0 2023-09-12T14:55:34.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d0ff9640 1 --2- 172.21.15.131:0/1308686777 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe8b4065b20 0x7fe8b4067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:34.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d37fe640 1 --2- 172.21.15.131:0/1308686777 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe8b4065b20 0x7fe8b4067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:34.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d0ff9640 1 -- 172.21.15.131:0/1308686777 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8cc083ca0 con 0x7fe8d40c4ee0 2023-09-12T14:55:34.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.268+0000 7fe8d37fe640 1 --2- 172.21.15.131:0/1308686777 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe8b4065b20 0x7fe8b4067fe0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40b0000 tx=0x7fe8c40a7040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:34.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.272+0000 7fe8d0ff9640 1 -- 172.21.15.131:0/1308686777 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fe8cc051680 con 0x7fe8d40c4ee0 2023-09-12T14:55:34.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.424+0000 7fe8d9b1c640 1 -- 172.21.15.131:0/1308686777 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8d415a650 con 0x7fe8d40c4ee0 2023-09-12T14:55:34.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.428+0000 7fe8d0ff9640 1 -- 172.21.15.131:0/1308686777 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fe8cc054ca0 con 0x7fe8d40c4ee0 2023-09-12T14:55:34.431 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:34.431 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:54:59.694778+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:54:55.689983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T14:54:57.029447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:55:34.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.428+0000 7fe8d9b1c640 1 -- 172.21.15.131:0/1308686777 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe8b4065b20 msgr2=0x7fe8b4067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:34.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.428+0000 7fe8d9b1c640 1 --2- 172.21.15.131:0/1308686777 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe8b4065b20 0x7fe8b4067fe0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40b0000 tx=0x7fe8c40a7040 comp rx=0 tx=0).stop 2023-09-12T14:55:34.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.428+0000 7fe8d9b1c640 1 -- 172.21.15.131:0/1308686777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8d40c4ee0 msgr2=0x7fe8d415a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:34.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.428+0000 7fe8d9b1c640 1 --2- 172.21.15.131:0/1308686777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8d40c4ee0 0x7fe8d415a980 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4089db0 tx=0x7fe8cc0019c0 comp rx=0 tx=0).stop 2023-09-12T14:55:34.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.428+0000 7fe8d9b1c640 1 -- 172.21.15.131:0/1308686777 shutdown_connections 2023-09-12T14:55:34.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.428+0000 7fe8d9b1c640 1 --2- 172.21.15.131:0/1308686777 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe8b4065b20 0x7fe8b4067fe0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:34.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.428+0000 7fe8d9b1c640 1 --2- 172.21.15.131:0/1308686777 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8d40c94a0 0x7fe8d40c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:34.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.428+0000 7fe8d9b1c640 1 --2- 172.21.15.131:0/1308686777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8d40c4ee0 0x7fe8d415a980 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:34.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.428+0000 7fe8d9b1c640 1 --2- 172.21.15.131:0/1308686777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8c00039b0 0x7fe8d40c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:34.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.428+0000 7fe8d9b1c640 1 -- 172.21.15.131:0/1308686777 >> 172.21.15.131:0/1308686777 conn(0x7fe8d409a200 msgr2=0x7fe8d40af010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:34.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.428+0000 7fe8d9b1c640 1 -- 172.21.15.131:0/1308686777 shutdown_connections 2023-09-12T14:55:34.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.428+0000 7fe8d9b1c640 1 -- 172.21.15.131:0/1308686777 wait complete. 2023-09-12T14:55:34.449 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T14:54:45.654004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T14:54:55.689983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T14:54:57.029447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-12T14:55:34.450 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T14:55:34.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.576+0000 7fb736ca8640 1 -- 172.21.15.131:0/3916045040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb73006b050 msgr2=0x7fb73006b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:34.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.576+0000 7fb736ca8640 1 --2- 172.21.15.131:0/3916045040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb73006b050 0x7fb73006b480 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb7300cc890 tx=0x7fb72801cb30 comp rx=0 tx=0).stop 2023-09-12T14:55:34.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.576+0000 7fb736ca8640 1 -- 172.21.15.131:0/3916045040 shutdown_connections 2023-09-12T14:55:34.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.576+0000 7fb736ca8640 1 --2- 172.21.15.131:0/3916045040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb73006b050 0x7fb73006b480 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:34.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.576+0000 7fb736ca8640 1 --2- 172.21.15.131:0/3916045040 >> v2:172.21.15.187:3300/0 conn(0x7fb730158180 0x7fb7300cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:34.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.576+0000 7fb736ca8640 1 -- 172.21.15.131:0/3916045040 >> 172.21.15.131:0/3916045040 conn(0x7fb730069840 msgr2=0x7fb7300cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:34.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.576+0000 7fb736ca8640 1 -- 172.21.15.131:0/3916045040 shutdown_connections 2023-09-12T14:55:34.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.576+0000 7fb736ca8640 1 -- 172.21.15.131:0/3916045040 wait complete. 2023-09-12T14:55:34.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.576+0000 7fb736ca8640 1 Processor -- start 2023-09-12T14:55:34.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.576+0000 7fb736ca8640 1 -- start start 2023-09-12T14:55:34.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.576+0000 7fb736ca8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb73006b050 0x7fb73015a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:34.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.576+0000 7fb736ca8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7300ab730 0x7fb73014cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:34.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.576+0000 7fb736ca8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb730158180 0x7fb73006f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:34.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.576+0000 7fb736ca8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73015aac0 con 0x7fb7300ab730 2023-09-12T14:55:34.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.576+0000 7fb736ca8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73014d650 con 0x7fb73006b050 2023-09-12T14:55:34.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.576+0000 7fb736ca8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb73014d930 con 0x7fb730158180 2023-09-12T14:55:34.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.576+0000 7fb72ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7300ab730 0x7fb73014cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:34.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.576+0000 7fb72ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7300ab730 0x7fb73014cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60024/0 (socket says 172.21.15.131:60024) 2023-09-12T14:55:34.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.576+0000 7fb72ffff640 1 -- 172.21.15.131:0/3582542045 learned_addr learned my addr 172.21.15.131:0/3582542045 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:34.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.576+0000 7fb735245640 1 --2- 172.21.15.131:0/3582542045 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb730158180 0x7fb73006f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:34.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.576+0000 7fb72ffff640 1 -- 172.21.15.131:0/3582542045 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb730158180 msgr2=0x7fb73006f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:34.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.576+0000 7fb72ffff640 1 --2- 172.21.15.131:0/3582542045 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb730158180 0x7fb73006f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:34.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.576+0000 7fb72ffff640 1 -- 172.21.15.131:0/3582542045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb73006b050 msgr2=0x7fb73015a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:34.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.576+0000 7fb72ffff640 1 --2- 172.21.15.131:0/3582542045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb73006b050 0x7fb73015a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:34.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.576+0000 7fb72ffff640 1 -- 172.21.15.131:0/3582542045 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb728009820 con 0x7fb7300ab730 2023-09-12T14:55:34.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.576+0000 7fb72ffff640 1 --2- 172.21.15.131:0/3582542045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7300ab730 0x7fb73014cc00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb72400d720 tx=0x7fb72400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:34.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.580+0000 7fb72d7fa640 1 -- 172.21.15.131:0/3582542045 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb724009c30 con 0x7fb7300ab730 2023-09-12T14:55:34.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.580+0000 7fb72d7fa640 1 -- 172.21.15.131:0/3582542045 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb724009e80 con 0x7fb7300ab730 2023-09-12T14:55:34.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.580+0000 7fb736ca8640 1 -- 172.21.15.131:0/3582542045 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb73014dc10 con 0x7fb7300ab730 2023-09-12T14:55:34.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.580+0000 7fb72d7fa640 1 -- 172.21.15.131:0/3582542045 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7240106a0 con 0x7fb7300ab730 2023-09-12T14:55:34.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.580+0000 7fb736ca8640 1 -- 172.21.15.131:0/3582542045 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb730159cd0 con 0x7fb7300ab730 2023-09-12T14:55:34.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.580+0000 7fb736ca8640 1 -- 172.21.15.131:0/3582542045 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb730073580 con 0x7fb7300ab730 2023-09-12T14:55:34.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.580+0000 7fb72d7fa640 1 -- 172.21.15.131:0/3582542045 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fb724010800 con 0x7fb7300ab730 2023-09-12T14:55:34.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.580+0000 7fb72d7fa640 1 --2- 172.21.15.131:0/3582542045 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb71c065b20 0x7fb71c067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:34.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.580+0000 7fb72d7fa640 1 -- 172.21.15.131:0/3582542045 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb724083750 con 0x7fb7300ab730 2023-09-12T14:55:34.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.580+0000 7fb734a44640 1 --2- 172.21.15.131:0/3582542045 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb71c065b20 0x7fb71c067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:34.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.580+0000 7fb734a44640 1 --2- 172.21.15.131:0/3582542045 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb71c065b20 0x7fb71c067fe0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb7200011f0 tx=0x7fb7200078f0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:34.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.584+0000 7fb72d7fa640 1 -- 172.21.15.131:0/3582542045 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fb724051130 con 0x7fb7300ab730 2023-09-12T14:55:34.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.736+0000 7fb736ca8640 1 -- 172.21.15.131:0/3582542045 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb7300cd4f0 con 0x7fb7300ab730 2023-09-12T14:55:34.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.740+0000 7fb72d7fa640 1 -- 172.21.15.131:0/3582542045 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb7240170a0 con 0x7fb7300ab730 2023-09-12T14:55:34.742 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-09-12T14:55:34.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.740+0000 7fb736ca8640 1 -- 172.21.15.131:0/3582542045 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb71c065b20 msgr2=0x7fb71c067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:34.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.740+0000 7fb736ca8640 1 --2- 172.21.15.131:0/3582542045 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb71c065b20 0x7fb71c067fe0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb7200011f0 tx=0x7fb7200078f0 comp rx=0 tx=0).stop 2023-09-12T14:55:34.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.740+0000 7fb736ca8640 1 -- 172.21.15.131:0/3582542045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7300ab730 msgr2=0x7fb73014cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:34.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.740+0000 7fb736ca8640 1 --2- 172.21.15.131:0/3582542045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7300ab730 0x7fb73014cc00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb72400d720 tx=0x7fb72400dbe0 comp rx=0 tx=0).stop 2023-09-12T14:55:34.745 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.740+0000 7fb736ca8640 1 -- 172.21.15.131:0/3582542045 shutdown_connections 2023-09-12T14:55:34.745 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.740+0000 7fb736ca8640 1 --2- 172.21.15.131:0/3582542045 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb71c065b20 0x7fb71c067fe0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:34.745 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.740+0000 7fb736ca8640 1 --2- 172.21.15.131:0/3582542045 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb730158180 0x7fb73006f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:34.745 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.740+0000 7fb736ca8640 1 --2- 172.21.15.131:0/3582542045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7300ab730 0x7fb73014cc00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:34.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.740+0000 7fb736ca8640 1 --2- 172.21.15.131:0/3582542045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb73006b050 0x7fb73015a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:34.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.740+0000 7fb736ca8640 1 -- 172.21.15.131:0/3582542045 >> 172.21.15.131:0/3582542045 conn(0x7fb730069840 msgr2=0x7fb7300adf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:34.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.740+0000 7fb736ca8640 1 -- 172.21.15.131:0/3582542045 shutdown_connections 2023-09-12T14:55:34.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.740+0000 7fb736ca8640 1 -- 172.21.15.131:0/3582542045 wait complete. 2023-09-12T14:55:34.760 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T14:55:34.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.880+0000 7f1164952640 1 -- 172.21.15.131:0/3781751905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1148005190 msgr2=0x7f114800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:34.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.880+0000 7f1164952640 1 --2- 172.21.15.131:0/3781751905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1148005190 0x7f114800b990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1158009e00 tx=0x7f115801d150 comp rx=0 tx=0).stop 2023-09-12T14:55:34.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.880+0000 7f1164952640 1 -- 172.21.15.131:0/3781751905 shutdown_connections 2023-09-12T14:55:34.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.880+0000 7f1164952640 1 --2- 172.21.15.131:0/3781751905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1148005190 0x7f114800b990 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:34.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.880+0000 7f1164952640 1 -- 172.21.15.131:0/3781751905 >> 172.21.15.131:0/3781751905 conn(0x7f1160069840 msgr2=0x7f11600cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:34.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.880+0000 7f1164952640 1 -- 172.21.15.131:0/3781751905 shutdown_connections 2023-09-12T14:55:34.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.880+0000 7f1164952640 1 -- 172.21.15.131:0/3781751905 wait complete. 2023-09-12T14:55:34.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.880+0000 7f1164952640 1 Processor -- start 2023-09-12T14:55:34.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.880+0000 7f1164952640 1 -- start start 2023-09-12T14:55:34.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.880+0000 7f1164952640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1148005190 0x7f1160156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:34.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.880+0000 7f1164952640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11601e0dd0 0x7f11601ec9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:34.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.880+0000 7f1164952640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11601e1250 0x7f11601ecf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:34.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.880+0000 7f1164952640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1160155db0 con 0x7f11601e1250 2023-09-12T14:55:34.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.880+0000 7f1164952640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11601ed580 con 0x7f1148005190 2023-09-12T14:55:34.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.880+0000 7f1164952640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11601ed860 con 0x7f11601e0dd0 2023-09-12T14:55:34.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.880+0000 7f115ed9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11601e1250 0x7f11601ecf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:34.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.880+0000 7f115ed9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11601e1250 0x7f11601ecf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60042/0 (socket says 172.21.15.131:60042) 2023-09-12T14:55:34.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.880+0000 7f115ed9d640 1 -- 172.21.15.131:0/2899035149 learned_addr learned my addr 172.21.15.131:0/2899035149 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:34.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.880+0000 7f115e59c640 1 --2- 172.21.15.131:0/2899035149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1148005190 0x7f1160156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:34.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.884+0000 7f115dd9b640 1 --2- 172.21.15.131:0/2899035149 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11601e0dd0 0x7f11601ec9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:34.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.884+0000 7f115ed9d640 1 -- 172.21.15.131:0/2899035149 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11601e0dd0 msgr2=0x7f11601ec9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:34.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.884+0000 7f115ed9d640 1 --2- 172.21.15.131:0/2899035149 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11601e0dd0 0x7f11601ec9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:34.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.884+0000 7f115ed9d640 1 -- 172.21.15.131:0/2899035149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1148005190 msgr2=0x7f1160156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:34.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.884+0000 7f115ed9d640 1 --2- 172.21.15.131:0/2899035149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1148005190 0x7f1160156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:34.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.884+0000 7f115ed9d640 1 -- 172.21.15.131:0/2899035149 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1160156050 con 0x7f11601e1250 2023-09-12T14:55:34.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.884+0000 7f115e59c640 1 --2- 172.21.15.131:0/2899035149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1148005190 0x7f1160156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:55:34.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.884+0000 7f115ed9d640 1 --2- 172.21.15.131:0/2899035149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11601e1250 0x7f11601ecf20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1158009e00 tx=0x7f1158003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:34.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.884+0000 7f114effd640 1 -- 172.21.15.131:0/2899035149 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1158009260 con 0x7f11601e1250 2023-09-12T14:55:34.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.884+0000 7f114effd640 1 -- 172.21.15.131:0/2899035149 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1158004320 con 0x7f11601e1250 2023-09-12T14:55:34.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.884+0000 7f1164952640 1 -- 172.21.15.131:0/2899035149 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1160156330 con 0x7f11601e1250 2023-09-12T14:55:34.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.884+0000 7f114effd640 1 -- 172.21.15.131:0/2899035149 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1158004710 con 0x7f11601e1250 2023-09-12T14:55:34.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.884+0000 7f1164952640 1 -- 172.21.15.131:0/2899035149 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1160148ff0 con 0x7f11601e1250 2023-09-12T14:55:34.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.884+0000 7f1164952640 1 -- 172.21.15.131:0/2899035149 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1160156d90 con 0x7f11601e1250 2023-09-12T14:55:34.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.884+0000 7f114effd640 1 -- 172.21.15.131:0/2899035149 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f11580c3460 con 0x7f11601e1250 2023-09-12T14:55:34.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.884+0000 7f114effd640 1 --2- 172.21.15.131:0/2899035149 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1140065b10 0x7f1140067fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:34.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.884+0000 7f114effd640 1 -- 172.21.15.131:0/2899035149 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11580a5e40 con 0x7f11601e1250 2023-09-12T14:55:34.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.884+0000 7f115e59c640 1 --2- 172.21.15.131:0/2899035149 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1140065b10 0x7f1140067fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:34.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.884+0000 7f115e59c640 1 --2- 172.21.15.131:0/2899035149 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1140065b10 0x7f1140067fd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f11500065b0 tx=0x7f1150006b10 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:34.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:34.888+0000 7f114effd640 1 -- 172.21.15.131:0/2899035149 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f11580ec2d0 con 0x7f11601e1250 2023-09-12T14:55:35.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.028+0000 7f1164952640 1 -- 172.21.15.131:0/2899035149 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1160156570 con 0x7f11601e1250 2023-09-12T14:55:35.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.028+0000 7f114effd640 1 -- 172.21.15.131:0/2899035149 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1160156570 con 0x7f11601e1250 2023-09-12T14:55:35.032 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-09-12T14:55:35.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.028+0000 7f1164952640 1 -- 172.21.15.131:0/2899035149 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1140065b10 msgr2=0x7f1140067fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:35.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.028+0000 7f1164952640 1 --2- 172.21.15.131:0/2899035149 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1140065b10 0x7f1140067fd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f11500065b0 tx=0x7f1150006b10 comp rx=0 tx=0).stop 2023-09-12T14:55:35.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.028+0000 7f1164952640 1 -- 172.21.15.131:0/2899035149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11601e1250 msgr2=0x7f11601ecf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:35.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.028+0000 7f1164952640 1 --2- 172.21.15.131:0/2899035149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11601e1250 0x7f11601ecf20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1158009e00 tx=0x7f1158003ef0 comp rx=0 tx=0).stop 2023-09-12T14:55:35.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.032+0000 7f1164952640 1 -- 172.21.15.131:0/2899035149 shutdown_connections 2023-09-12T14:55:35.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.032+0000 7f1164952640 1 --2- 172.21.15.131:0/2899035149 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1140065b10 0x7f1140067fd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:35.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.032+0000 7f1164952640 1 --2- 172.21.15.131:0/2899035149 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11601e0dd0 0x7f11601ec9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:35.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.032+0000 7f1164952640 1 --2- 172.21.15.131:0/2899035149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11601e1250 0x7f11601ecf20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:35.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.032+0000 7f1164952640 1 --2- 172.21.15.131:0/2899035149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1148005190 0x7f1160156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:35.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.032+0000 7f1164952640 1 -- 172.21.15.131:0/2899035149 >> 172.21.15.131:0/2899035149 conn(0x7f1160069840 msgr2=0x7f1160154220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:35.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.032+0000 7f1164952640 1 -- 172.21.15.131:0/2899035149 shutdown_connections 2023-09-12T14:55:35.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.032+0000 7f1164952640 1 -- 172.21.15.131:0/2899035149 wait complete. 2023-09-12T14:55:35.049 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T14:55:35.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.164+0000 7f5b0be0f640 1 -- 172.21.15.131:0/3053709288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5af0004ac0 msgr2=0x7f5af0004ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:35.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.164+0000 7f5b0be0f640 1 --2- 172.21.15.131:0/3053709288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5af0004ac0 0x7f5af0004ee0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5b00001680 tx=0x7f5b0001c730 comp rx=0 tx=0).stop 2023-09-12T14:55:35.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.164+0000 7f5b0a3ac640 1 -- 172.21.15.131:0/3053709288 reap_dead start 2023-09-12T14:55:35.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.164+0000 7f5b0be0f640 1 -- 172.21.15.131:0/3053709288 shutdown_connections 2023-09-12T14:55:35.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.164+0000 7f5b0be0f640 1 -- 172.21.15.131:0/3053709288 >> 172.21.15.131:0/3053709288 conn(0x7f5b04069840 msgr2=0x7f5b040cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:35.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.164+0000 7f5b0be0f640 1 -- 172.21.15.131:0/3053709288 shutdown_connections 2023-09-12T14:55:35.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.164+0000 7f5b0be0f640 1 -- 172.21.15.131:0/3053709288 wait complete. 2023-09-12T14:55:35.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5b0be0f640 1 Processor -- start 2023-09-12T14:55:35.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5b0be0f640 1 -- start start 2023-09-12T14:55:35.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5b0be0f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5af0004ac0 0x7f5b04147f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:35.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5b0be0f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b04148820 0x7f5b040aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:35.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5b0be0f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b041f3e50 0x7f5b0415c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:35.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5b0be0f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b04159e80 con 0x7f5b041f3e50 2023-09-12T14:55:35.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5b0be0f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b041f4d80 con 0x7f5af0004ac0 2023-09-12T14:55:35.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5b0be0f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b041f47c0 con 0x7f5b04148820 2023-09-12T14:55:35.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5b0a3ac640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b041f3e50 0x7f5b0415c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:35.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5b0a3ac640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b041f3e50 0x7f5b0415c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60066/0 (socket says 172.21.15.131:60066) 2023-09-12T14:55:35.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5b0a3ac640 1 -- 172.21.15.131:0/2379873070 learned_addr learned my addr 172.21.15.131:0/2379873070 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:35.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5b09bab640 1 --2- 172.21.15.131:0/2379873070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5af0004ac0 0x7f5b04147f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:35.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5b0a3ac640 1 -- 172.21.15.131:0/2379873070 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b04148820 msgr2=0x7f5b040aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:35.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5b093aa640 1 --2- 172.21.15.131:0/2379873070 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b04148820 0x7f5b040aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:35.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5b0a3ac640 1 --2- 172.21.15.131:0/2379873070 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b04148820 0x7f5b040aeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:35.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5b0a3ac640 1 -- 172.21.15.131:0/2379873070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5af0004ac0 msgr2=0x7f5b04147f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:35.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5b0a3ac640 1 --2- 172.21.15.131:0/2379873070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5af0004ac0 0x7f5b04147f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:35.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5b0a3ac640 1 -- 172.21.15.131:0/2379873070 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b041f4aa0 con 0x7f5b041f3e50 2023-09-12T14:55:35.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5b09bab640 1 --2- 172.21.15.131:0/2379873070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5af0004ac0 0x7f5b04147f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:35.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5b093aa640 1 --2- 172.21.15.131:0/2379873070 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b04148820 0x7f5b040aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:35.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5b0a3ac640 1 --2- 172.21.15.131:0/2379873070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b041f3e50 0x7f5b0415c9c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5afc001570 tx=0x7f5afc00cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:35.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5afa7fc640 1 -- 172.21.15.131:0/2379873070 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc00aa20 con 0x7f5b041f3e50 2023-09-12T14:55:35.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5afa7fc640 1 -- 172.21.15.131:0/2379873070 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5afc00aba0 con 0x7f5b041f3e50 2023-09-12T14:55:35.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5b0be0f640 1 -- 172.21.15.131:0/2379873070 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b040ae6c0 con 0x7f5b041f3e50 2023-09-12T14:55:35.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5afa7fc640 1 -- 172.21.15.131:0/2379873070 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc007070 con 0x7f5b041f3e50 2023-09-12T14:55:35.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5b0be0f640 1 -- 172.21.15.131:0/2379873070 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b04148480 con 0x7f5b041f3e50 2023-09-12T14:55:35.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5b0be0f640 1 -- 172.21.15.131:0/2379873070 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b04073580 con 0x7f5b041f3e50 2023-09-12T14:55:35.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.168+0000 7f5afa7fc640 1 -- 172.21.15.131:0/2379873070 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f5afc0071d0 con 0x7f5b041f3e50 2023-09-12T14:55:35.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.172+0000 7f5afa7fc640 1 --2- 172.21.15.131:0/2379873070 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5ae8065b20 0x7f5ae8067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:35.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.172+0000 7f5afa7fc640 1 -- 172.21.15.131:0/2379873070 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5afc082fc0 con 0x7f5b041f3e50 2023-09-12T14:55:35.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.172+0000 7f5b09bab640 1 --2- 172.21.15.131:0/2379873070 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5ae8065b20 0x7f5ae8067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:35.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.172+0000 7f5b09bab640 1 --2- 172.21.15.131:0/2379873070 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5ae8065b20 0x7f5ae8067fe0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5af00068c0 tx=0x7f5b000a8040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:35.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.176+0000 7f5afa7fc640 1 -- 172.21.15.131:0/2379873070 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f5afc0509a0 con 0x7f5b041f3e50 2023-09-12T14:55:35.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.316+0000 7f5b0be0f640 1 -- 172.21.15.131:0/2379873070 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5b040aeb70 con 0x7f5b041f3e50 2023-09-12T14:55:35.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.316+0000 7f5afa7fc640 1 -- 172.21.15.131:0/2379873070 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5afc017020 con 0x7f5b041f3e50 2023-09-12T14:55:35.321 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-09-12T14:55:35.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.320+0000 7f5b0be0f640 1 -- 172.21.15.131:0/2379873070 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5ae8065b20 msgr2=0x7f5ae8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:35.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.320+0000 7f5b0be0f640 1 --2- 172.21.15.131:0/2379873070 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5ae8065b20 0x7f5ae8067fe0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5af00068c0 tx=0x7f5b000a8040 comp rx=0 tx=0).stop 2023-09-12T14:55:35.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.320+0000 7f5b0be0f640 1 -- 172.21.15.131:0/2379873070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b041f3e50 msgr2=0x7f5b0415c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:35.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.320+0000 7f5b0be0f640 1 --2- 172.21.15.131:0/2379873070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b041f3e50 0x7f5b0415c9c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5afc001570 tx=0x7f5afc00cd00 comp rx=0 tx=0).stop 2023-09-12T14:55:35.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.320+0000 7f5b0be0f640 1 -- 172.21.15.131:0/2379873070 shutdown_connections 2023-09-12T14:55:35.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.320+0000 7f5b0be0f640 1 --2- 172.21.15.131:0/2379873070 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5ae8065b20 0x7f5ae8067fe0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:35.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.320+0000 7f5b0be0f640 1 --2- 172.21.15.131:0/2379873070 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b04148820 0x7f5b040aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:35.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.320+0000 7f5b0be0f640 1 --2- 172.21.15.131:0/2379873070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b041f3e50 0x7f5b0415c9c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:35.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.320+0000 7f5b0be0f640 1 --2- 172.21.15.131:0/2379873070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5af0004ac0 0x7f5b04147f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:35.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.320+0000 7f5b0be0f640 1 -- 172.21.15.131:0/2379873070 >> 172.21.15.131:0/2379873070 conn(0x7f5b04069840 msgr2=0x7f5b04151e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:35.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.320+0000 7f5b0be0f640 1 -- 172.21.15.131:0/2379873070 shutdown_connections 2023-09-12T14:55:35.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.320+0000 7f5b0be0f640 1 -- 172.21.15.131:0/2379873070 wait complete. 2023-09-12T14:55:35.338 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-12T14:55:35.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.460+0000 7f93879f5640 1 -- 172.21.15.131:0/4088100577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93680039b0 msgr2=0x7f9368003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:35.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.460+0000 7f93879f5640 1 --2- 172.21.15.131:0/4088100577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93680039b0 0x7f9368003dd0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f937000c5c0 tx=0x7f937001d340 comp rx=0 tx=0).stop 2023-09-12T14:55:35.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.460+0000 7f9385f92640 1 -- 172.21.15.131:0/4088100577 reap_dead start 2023-09-12T14:55:35.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.460+0000 7f93879f5640 1 -- 172.21.15.131:0/4088100577 shutdown_connections 2023-09-12T14:55:35.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.460+0000 7f93879f5640 1 -- 172.21.15.131:0/4088100577 >> 172.21.15.131:0/4088100577 conn(0x7f9380069840 msgr2=0x7f93800cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:35.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.460+0000 7f93879f5640 1 -- 172.21.15.131:0/4088100577 shutdown_connections 2023-09-12T14:55:35.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.460+0000 7f93879f5640 1 -- 172.21.15.131:0/4088100577 wait complete. 2023-09-12T14:55:35.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.460+0000 7f93879f5640 1 Processor -- start 2023-09-12T14:55:35.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.460+0000 7f93879f5640 1 -- start start 2023-09-12T14:55:35.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.464+0000 7f93879f5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93680039b0 0x7f9380134250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:35.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.464+0000 7f93879f5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9380148bd0 0x7f9380143c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:35.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.464+0000 7f93879f5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93801441d0 0x7f938014eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:35.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.464+0000 7f93879f5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938014aa00 con 0x7f93801441d0 2023-09-12T14:55:35.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.464+0000 7f93879f5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938014ace0 con 0x7f93680039b0 2023-09-12T14:55:35.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.464+0000 7f93879f5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f938014afc0 con 0x7f9380148bd0 2023-09-12T14:55:35.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.464+0000 7f9385f92640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93801441d0 0x7f938014eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:35.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.464+0000 7f9385f92640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93801441d0 0x7f938014eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60086/0 (socket says 172.21.15.131:60086) 2023-09-12T14:55:35.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.464+0000 7f9385f92640 1 -- 172.21.15.131:0/3494739641 learned_addr learned my addr 172.21.15.131:0/3494739641 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:35.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.464+0000 7f9385791640 1 --2- 172.21.15.131:0/3494739641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93680039b0 0x7f9380134250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:35.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.464+0000 7f9384f90640 1 --2- 172.21.15.131:0/3494739641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9380148bd0 0x7f9380143c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:35.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.464+0000 7f9385f92640 1 -- 172.21.15.131:0/3494739641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9380148bd0 msgr2=0x7f9380143c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:35.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.464+0000 7f9385f92640 1 --2- 172.21.15.131:0/3494739641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9380148bd0 0x7f9380143c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:35.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.464+0000 7f9385f92640 1 -- 172.21.15.131:0/3494739641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93680039b0 msgr2=0x7f9380134250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:35.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.464+0000 7f9385f92640 1 --2- 172.21.15.131:0/3494739641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93680039b0 0x7f9380134250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:35.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.464+0000 7f9385f92640 1 -- 172.21.15.131:0/3494739641 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938014b2a0 con 0x7f93801441d0 2023-09-12T14:55:35.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.464+0000 7f9385f92640 1 --2- 172.21.15.131:0/3494739641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93801441d0 0x7f938014eea0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f93800cd8d0 tx=0x7f937800ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:35.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.464+0000 7f937dffb640 1 -- 172.21.15.131:0/3494739641 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378018070 con 0x7f93801441d0 2023-09-12T14:55:35.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.464+0000 7f937dffb640 1 -- 172.21.15.131:0/3494739641 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9378005110 con 0x7f93801441d0 2023-09-12T14:55:35.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.464+0000 7f93879f5640 1 -- 172.21.15.131:0/3494739641 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f938014b580 con 0x7f93801441d0 2023-09-12T14:55:35.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.464+0000 7f937dffb640 1 -- 172.21.15.131:0/3494739641 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378005500 con 0x7f93801441d0 2023-09-12T14:55:35.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.464+0000 7f93879f5640 1 -- 172.21.15.131:0/3494739641 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f938014a780 con 0x7f93801441d0 2023-09-12T14:55:35.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.464+0000 7f93879f5640 1 -- 172.21.15.131:0/3494739641 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93800ab7e0 con 0x7f93801441d0 2023-09-12T14:55:35.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.464+0000 7f937dffb640 1 -- 172.21.15.131:0/3494739641 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f93780047b0 con 0x7f93801441d0 2023-09-12T14:55:35.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.464+0000 7f937dffb640 1 --2- 172.21.15.131:0/3494739641 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f936c065b50 0x7f936c068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:35.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.464+0000 7f9385791640 1 --2- 172.21.15.131:0/3494739641 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f936c065b50 0x7f936c068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:35.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.464+0000 7f937dffb640 1 -- 172.21.15.131:0/3494739641 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9378082e20 con 0x7f93801441d0 2023-09-12T14:55:35.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.468+0000 7f9385791640 1 --2- 172.21.15.131:0/3494739641 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f936c065b50 0x7f936c068010 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f93700ac000 tx=0x7f937001a0e0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:35.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.468+0000 7f937dffb640 1 -- 172.21.15.131:0/3494739641 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f9378050770 con 0x7f93801441d0 2023-09-12T14:55:35.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.640+0000 7f93879f5640 1 -- 172.21.15.131:0/3494739641 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9380073580 con 0x7f93801441d0 2023-09-12T14:55:35.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.644+0000 7f937dffb640 1 -- 172.21.15.131:0/3494739641 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9378053d90 con 0x7f93801441d0 2023-09-12T14:55:35.646 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2023-09-12T14:55:35.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.644+0000 7f93879f5640 1 -- 172.21.15.131:0/3494739641 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f936c065b50 msgr2=0x7f936c068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:35.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.644+0000 7f93879f5640 1 --2- 172.21.15.131:0/3494739641 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f936c065b50 0x7f936c068010 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f93700ac000 tx=0x7f937001a0e0 comp rx=0 tx=0).stop 2023-09-12T14:55:35.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.644+0000 7f93879f5640 1 -- 172.21.15.131:0/3494739641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93801441d0 msgr2=0x7f938014eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:35.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.644+0000 7f93879f5640 1 --2- 172.21.15.131:0/3494739641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93801441d0 0x7f938014eea0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f93800cd8d0 tx=0x7f937800ea80 comp rx=0 tx=0).stop 2023-09-12T14:55:35.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.644+0000 7f93879f5640 1 -- 172.21.15.131:0/3494739641 shutdown_connections 2023-09-12T14:55:35.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.644+0000 7f93879f5640 1 --2- 172.21.15.131:0/3494739641 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f936c065b50 0x7f936c068010 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:35.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.644+0000 7f93879f5640 1 --2- 172.21.15.131:0/3494739641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9380148bd0 0x7f9380143c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:35.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.644+0000 7f93879f5640 1 --2- 172.21.15.131:0/3494739641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93801441d0 0x7f938014eea0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:35.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.644+0000 7f93879f5640 1 --2- 172.21.15.131:0/3494739641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93680039b0 0x7f9380134250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:35.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.644+0000 7f93879f5640 1 -- 172.21.15.131:0/3494739641 >> 172.21.15.131:0/3494739641 conn(0x7f9380069840 msgr2=0x7f9380158110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:35.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.644+0000 7f93879f5640 1 -- 172.21.15.131:0/3494739641 shutdown_connections 2023-09-12T14:55:35.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.644+0000 7f93879f5640 1 -- 172.21.15.131:0/3494739641 wait complete. 2023-09-12T14:55:35.664 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:55:35.665 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-12T14:55:35.672 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:55:35.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb95b1da640 1 -- 172.21.15.131:0/3302983035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9540d1680 msgr2=0x7fb9540b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:35.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb95b1da640 1 --2- 172.21.15.131:0/3302983035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9540d1680 0x7fb9540b0750 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb948009900 tx=0x7fb94801c850 comp rx=0 tx=0).stop 2023-09-12T14:55:35.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb95b1da640 1 -- 172.21.15.131:0/3302983035 shutdown_connections 2023-09-12T14:55:35.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb95b1da640 1 --2- 172.21.15.131:0/3302983035 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb95415e9f0 0x7fb9540b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:35.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb95b1da640 1 --2- 172.21.15.131:0/3302983035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9540d1680 0x7fb9540b0750 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:35.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb95b1da640 1 -- 172.21.15.131:0/3302983035 >> 172.21.15.131:0/3302983035 conn(0x7fb95409a200 msgr2=0x7fb9540d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:35.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb95b1da640 1 -- 172.21.15.131:0/3302983035 shutdown_connections 2023-09-12T14:55:35.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb95b1da640 1 -- 172.21.15.131:0/3302983035 wait complete. 2023-09-12T14:55:35.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb95b1da640 1 Processor -- start 2023-09-12T14:55:35.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb95b1da640 1 -- start start 2023-09-12T14:55:35.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb95b1da640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9540cf2d0 0x7fb9541565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:35.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb95b1da640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9540d1680 0x7fb9540c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:35.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb95b1da640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb95415e9f0 0x7fb9540c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:35.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb95b1da640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb954155540 con 0x7fb9540d1680 2023-09-12T14:55:35.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb95b1da640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb954155820 con 0x7fb9540cf2d0 2023-09-12T14:55:35.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb95b1da640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb954155b00 con 0x7fb95415e9f0 2023-09-12T14:55:35.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb953fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9540d1680 0x7fb9540c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:35.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb953fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9540d1680 0x7fb9540c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60098/0 (socket says 172.21.15.131:60098) 2023-09-12T14:55:35.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb953fff640 1 -- 172.21.15.131:0/382059955 learned_addr learned my addr 172.21.15.131:0/382059955 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:35.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb958f76640 1 --2- 172.21.15.131:0/382059955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9540cf2d0 0x7fb9541565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:35.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb959777640 1 --2- 172.21.15.131:0/382059955 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb95415e9f0 0x7fb9540c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:35.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb953fff640 1 -- 172.21.15.131:0/382059955 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb95415e9f0 msgr2=0x7fb9540c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:35.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb953fff640 1 --2- 172.21.15.131:0/382059955 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb95415e9f0 0x7fb9540c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:35.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb953fff640 1 -- 172.21.15.131:0/382059955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9540cf2d0 msgr2=0x7fb9541565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:35.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb953fff640 1 --2- 172.21.15.131:0/382059955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9540cf2d0 0x7fb9541565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:35.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb953fff640 1 -- 172.21.15.131:0/382059955 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb948009560 con 0x7fb9540d1680 2023-09-12T14:55:35.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb958f76640 1 --2- 172.21.15.131:0/382059955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9540cf2d0 0x7fb9541565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:35.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb953fff640 1 --2- 172.21.15.131:0/382059955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9540d1680 0x7fb9540c8040 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb948000c10 tx=0x7fb9480a57c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:35.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb9517fa640 1 -- 172.21.15.131:0/382059955 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9480052f0 con 0x7fb9540d1680 2023-09-12T14:55:35.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb9517fa640 1 -- 172.21.15.131:0/382059955 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb948005560 con 0x7fb9540d1680 2023-09-12T14:55:35.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.796+0000 7fb9517fa640 1 -- 172.21.15.131:0/382059955 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb948004930 con 0x7fb9540d1680 2023-09-12T14:55:35.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.800+0000 7fb95b1da640 1 -- 172.21.15.131:0/382059955 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb954155de0 con 0x7fb9540d1680 2023-09-12T14:55:35.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.800+0000 7fb95b1da640 1 -- 172.21.15.131:0/382059955 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb954156b00 con 0x7fb9540d1680 2023-09-12T14:55:35.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.800+0000 7fb95b1da640 1 -- 172.21.15.131:0/382059955 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb954073580 con 0x7fb9540d1680 2023-09-12T14:55:35.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.800+0000 7fb9517fa640 1 -- 172.21.15.131:0/382059955 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fb9480b4040 con 0x7fb9540d1680 2023-09-12T14:55:35.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.800+0000 7fb9517fa640 1 --2- 172.21.15.131:0/382059955 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb940065b20 0x7fb940067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:35.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.800+0000 7fb9517fa640 1 -- 172.21.15.131:0/382059955 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb94811d470 con 0x7fb9540d1680 2023-09-12T14:55:35.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.800+0000 7fb958f76640 1 --2- 172.21.15.131:0/382059955 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb940065b20 0x7fb940067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:35.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.800+0000 7fb958f76640 1 --2- 172.21.15.131:0/382059955 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb940065b20 0x7fb940067fe0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb944003e40 tx=0x7fb944009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:35.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.804+0000 7fb9517fa640 1 -- 172.21.15.131:0/382059955 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fb9480eae50 con 0x7fb9540d1680 2023-09-12T14:55:35.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.960+0000 7fb95b1da640 1 -- 172.21.15.131:0/382059955 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9540afee0 con 0x7fb9540d1680 2023-09-12T14:55:35.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.960+0000 7fb9517fa640 1 -- 172.21.15.131:0/382059955 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fb9480ee470 con 0x7fb9540d1680 2023-09-12T14:55:35.965 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:35.966 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:54:59.694778+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:54:55.689983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T14:54:57.029447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:55:35.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.964+0000 7fb95b1da640 1 -- 172.21.15.131:0/382059955 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb940065b20 msgr2=0x7fb940067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:35.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.964+0000 7fb95b1da640 1 --2- 172.21.15.131:0/382059955 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb940065b20 0x7fb940067fe0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb944003e40 tx=0x7fb944009040 comp rx=0 tx=0).stop 2023-09-12T14:55:35.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.964+0000 7fb95b1da640 1 -- 172.21.15.131:0/382059955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9540d1680 msgr2=0x7fb9540c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:35.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.964+0000 7fb95b1da640 1 --2- 172.21.15.131:0/382059955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9540d1680 0x7fb9540c8040 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb948000c10 tx=0x7fb9480a57c0 comp rx=0 tx=0).stop 2023-09-12T14:55:35.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.964+0000 7fb95b1da640 1 -- 172.21.15.131:0/382059955 shutdown_connections 2023-09-12T14:55:35.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.964+0000 7fb95b1da640 1 --2- 172.21.15.131:0/382059955 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb940065b20 0x7fb940067fe0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:35.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.964+0000 7fb95b1da640 1 --2- 172.21.15.131:0/382059955 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb95415e9f0 0x7fb9540c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:35.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.964+0000 7fb95b1da640 1 --2- 172.21.15.131:0/382059955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9540d1680 0x7fb9540c8040 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:35.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.964+0000 7fb95b1da640 1 --2- 172.21.15.131:0/382059955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9540cf2d0 0x7fb9541565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:35.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.964+0000 7fb95b1da640 1 -- 172.21.15.131:0/382059955 >> 172.21.15.131:0/382059955 conn(0x7fb95409a200 msgr2=0x7fb9540cbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:35.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.964+0000 7fb95b1da640 1 -- 172.21.15.131:0/382059955 shutdown_connections 2023-09-12T14:55:35.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:35.964+0000 7fb95b1da640 1 -- 172.21.15.131:0/382059955 wait complete. 2023-09-12T14:55:35.984 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T14:54:45.654004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T14:54:55.689983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T14:54:57.029447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-12T14:55:35.985 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T14:55:36.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.112+0000 7f3862081640 1 -- 172.21.15.131:0/4177322531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f385c06c060 msgr2=0x7f385c1626d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:36.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.112+0000 7f3862081640 1 --2- 172.21.15.131:0/4177322531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f385c06c060 0x7f385c1626d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3854009bc0 tx=0x7f385401ca80 comp rx=0 tx=0).stop 2023-09-12T14:55:36.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.112+0000 7f3862081640 1 -- 172.21.15.131:0/4177322531 shutdown_connections 2023-09-12T14:55:36.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.112+0000 7f3862081640 1 --2- 172.21.15.131:0/4177322531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f385c06c060 0x7f385c1626d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:36.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.112+0000 7f3862081640 1 -- 172.21.15.131:0/4177322531 >> 172.21.15.131:0/4177322531 conn(0x7f385c069840 msgr2=0x7f385c0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:36.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.112+0000 7f3862081640 1 -- 172.21.15.131:0/4177322531 shutdown_connections 2023-09-12T14:55:36.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.112+0000 7f3862081640 1 -- 172.21.15.131:0/4177322531 wait complete. 2023-09-12T14:55:36.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.112+0000 7f3862081640 1 Processor -- start 2023-09-12T14:55:36.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.112+0000 7f3862081640 1 -- start start 2023-09-12T14:55:36.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.112+0000 7f3862081640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f385c06c060 0x7f385c14b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:36.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.112+0000 7f3862081640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f385c0ab730 0x7f385c147250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:36.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.112+0000 7f3862081640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f385c158180 0x7f385c1f1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:36.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.112+0000 7f3862081640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c06fd60 con 0x7f385c06c060 2023-09-12T14:55:36.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.112+0000 7f3862081640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c149000 con 0x7f385c0ab730 2023-09-12T14:55:36.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.112+0000 7f3862081640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385c14b9c0 con 0x7f385c158180 2023-09-12T14:55:36.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.112+0000 7f385b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f385c06c060 0x7f385c14b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:36.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.112+0000 7f385b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f385c06c060 0x7f385c14b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60108/0 (socket says 172.21.15.131:60108) 2023-09-12T14:55:36.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.112+0000 7f385b7fe640 1 -- 172.21.15.131:0/1416457509 learned_addr learned my addr 172.21.15.131:0/1416457509 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:36.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.116+0000 7f385b7fe640 1 -- 172.21.15.131:0/1416457509 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f385c158180 msgr2=0x7f385c1f1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:36.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.116+0000 7f385bfff640 1 --2- 172.21.15.131:0/1416457509 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f385c158180 0x7f385c1f1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:36.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.116+0000 7f385affd640 1 --2- 172.21.15.131:0/1416457509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f385c0ab730 0x7f385c147250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:36.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.116+0000 7f385b7fe640 1 --2- 172.21.15.131:0/1416457509 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f385c158180 0x7f385c1f1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:36.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.116+0000 7f385b7fe640 1 -- 172.21.15.131:0/1416457509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f385c0ab730 msgr2=0x7f385c147250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:36.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.116+0000 7f385b7fe640 1 --2- 172.21.15.131:0/1416457509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f385c0ab730 0x7f385c147250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:36.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.116+0000 7f385b7fe640 1 -- 172.21.15.131:0/1416457509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3854009820 con 0x7f385c06c060 2023-09-12T14:55:36.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.116+0000 7f385affd640 1 --2- 172.21.15.131:0/1416457509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f385c0ab730 0x7f385c147250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:36.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.116+0000 7f385b7fe640 1 --2- 172.21.15.131:0/1416457509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f385c06c060 0x7f385c14b480 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f385000c450 tx=0x7f385000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:36.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.116+0000 7f3843fff640 1 -- 172.21.15.131:0/1416457509 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3850012830 con 0x7f385c06c060 2023-09-12T14:55:36.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.116+0000 7f3843fff640 1 -- 172.21.15.131:0/1416457509 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3850012aa0 con 0x7f385c06c060 2023-09-12T14:55:36.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.116+0000 7f3862081640 1 -- 172.21.15.131:0/1416457509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f385c14bc40 con 0x7f385c06c060 2023-09-12T14:55:36.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.116+0000 7f3843fff640 1 -- 172.21.15.131:0/1416457509 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3850011380 con 0x7f385c06c060 2023-09-12T14:55:36.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.116+0000 7f3862081640 1 -- 172.21.15.131:0/1416457509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f385c148c90 con 0x7f385c06c060 2023-09-12T14:55:36.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.116+0000 7f3862081640 1 -- 172.21.15.131:0/1416457509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f385c073580 con 0x7f385c06c060 2023-09-12T14:55:36.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.116+0000 7f3843fff640 1 -- 172.21.15.131:0/1416457509 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f38500114e0 con 0x7f385c06c060 2023-09-12T14:55:36.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.116+0000 7f3843fff640 1 --2- 172.21.15.131:0/1416457509 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3848065b20 0x7f3848067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:36.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.116+0000 7f385affd640 1 --2- 172.21.15.131:0/1416457509 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3848065b20 0x7f3848067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:36.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.116+0000 7f3843fff640 1 -- 172.21.15.131:0/1416457509 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3850083360 con 0x7f385c06c060 2023-09-12T14:55:36.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.116+0000 7f385affd640 1 --2- 172.21.15.131:0/1416457509 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3848065b20 0x7f3848067fe0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f385c0cd660 tx=0x7f384c009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:36.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.120+0000 7f3843fff640 1 -- 172.21.15.131:0/1416457509 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f3850050d40 con 0x7f385c06c060 2023-09-12T14:55:36.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.272+0000 7f3862081640 1 -- 172.21.15.131:0/1416457509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f385c0ac470 con 0x7f385c06c060 2023-09-12T14:55:36.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.276+0000 7f3843fff640 1 -- 172.21.15.131:0/1416457509 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3850054360 con 0x7f385c06c060 2023-09-12T14:55:36.278 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-09-12T14:55:36.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.276+0000 7f3862081640 1 -- 172.21.15.131:0/1416457509 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3848065b20 msgr2=0x7f3848067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:36.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.276+0000 7f3862081640 1 --2- 172.21.15.131:0/1416457509 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3848065b20 0x7f3848067fe0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f385c0cd660 tx=0x7f384c009040 comp rx=0 tx=0).stop 2023-09-12T14:55:36.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.276+0000 7f3862081640 1 -- 172.21.15.131:0/1416457509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f385c06c060 msgr2=0x7f385c14b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:36.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.276+0000 7f3862081640 1 --2- 172.21.15.131:0/1416457509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f385c06c060 0x7f385c14b480 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f385000c450 tx=0x7f385000c910 comp rx=0 tx=0).stop 2023-09-12T14:55:36.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.276+0000 7f3862081640 1 -- 172.21.15.131:0/1416457509 shutdown_connections 2023-09-12T14:55:36.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.276+0000 7f3862081640 1 --2- 172.21.15.131:0/1416457509 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3848065b20 0x7f3848067fe0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:36.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.276+0000 7f3862081640 1 --2- 172.21.15.131:0/1416457509 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f385c158180 0x7f385c1f1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:36.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.276+0000 7f3862081640 1 --2- 172.21.15.131:0/1416457509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f385c0ab730 0x7f385c147250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:36.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.276+0000 7f3862081640 1 --2- 172.21.15.131:0/1416457509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f385c06c060 0x7f385c14b480 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:36.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.276+0000 7f3862081640 1 -- 172.21.15.131:0/1416457509 >> 172.21.15.131:0/1416457509 conn(0x7f385c069840 msgr2=0x7f385c0abb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:36.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.276+0000 7f3862081640 1 -- 172.21.15.131:0/1416457509 shutdown_connections 2023-09-12T14:55:36.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.276+0000 7f3862081640 1 -- 172.21.15.131:0/1416457509 wait complete. 2023-09-12T14:55:36.296 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T14:55:36.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.408+0000 7f5e13c42640 1 -- 172.21.15.131:0/1364773022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df8005190 msgr2=0x7f5df800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:36.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.408+0000 7f5e13c42640 1 --2- 172.21.15.131:0/1364773022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df8005190 0x7f5df800b990 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5e04009ee0 tx=0x7f5e0401d610 comp rx=0 tx=0).stop 2023-09-12T14:55:36.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.408+0000 7f5e13c42640 1 -- 172.21.15.131:0/1364773022 shutdown_connections 2023-09-12T14:55:36.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.408+0000 7f5e13c42640 1 --2- 172.21.15.131:0/1364773022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df8005190 0x7f5df800b990 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:36.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.408+0000 7f5e13c42640 1 -- 172.21.15.131:0/1364773022 >> 172.21.15.131:0/1364773022 conn(0x7f5e0c098040 msgr2=0x7f5e0c0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:36.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.408+0000 7f5e13c42640 1 -- 172.21.15.131:0/1364773022 shutdown_connections 2023-09-12T14:55:36.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.408+0000 7f5e13c42640 1 -- 172.21.15.131:0/1364773022 wait complete. 2023-09-12T14:55:36.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.412+0000 7f5e13c42640 1 Processor -- start 2023-09-12T14:55:36.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.412+0000 7f5e13c42640 1 -- start start 2023-09-12T14:55:36.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.412+0000 7f5e13c42640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5df8005190 0x7f5e0c0b4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:36.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.412+0000 7f5e13c42640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e0c15aa40 0x7f5e0c0c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:36.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.412+0000 7f5e13c42640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e0c147430 0x7f5e0c0b1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:36.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.412+0000 7f5e13c42640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c0c8800 con 0x7f5e0c15aa40 2023-09-12T14:55:36.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.412+0000 7f5e13c42640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c0c7220 con 0x7f5e0c147430 2023-09-12T14:55:36.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.412+0000 7f5e13c42640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c0c7500 con 0x7f5df8005190 2023-09-12T14:55:36.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.412+0000 7f5e111dd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e0c15aa40 0x7f5e0c0c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:36.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.412+0000 7f5e111dd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e0c15aa40 0x7f5e0c0c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60122/0 (socket says 172.21.15.131:60122) 2023-09-12T14:55:36.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.412+0000 7f5e111dd640 1 -- 172.21.15.131:0/977847993 learned_addr learned my addr 172.21.15.131:0/977847993 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:36.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.412+0000 7f5e119de640 1 --2- 172.21.15.131:0/977847993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5df8005190 0x7f5e0c0b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:36.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.412+0000 7f5e111dd640 1 -- 172.21.15.131:0/977847993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5df8005190 msgr2=0x7f5e0c0b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:36.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.412+0000 7f5e111dd640 1 --2- 172.21.15.131:0/977847993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5df8005190 0x7f5e0c0b4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:36.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.412+0000 7f5e111dd640 1 -- 172.21.15.131:0/977847993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e0c147430 msgr2=0x7f5e0c0b1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:36.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.412+0000 7f5e111dd640 1 --2- 172.21.15.131:0/977847993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e0c147430 0x7f5e0c0b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:36.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.412+0000 7f5e111dd640 1 -- 172.21.15.131:0/977847993 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e0c0c77e0 con 0x7f5e0c15aa40 2023-09-12T14:55:36.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.412+0000 7f5e119de640 1 --2- 172.21.15.131:0/977847993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5df8005190 0x7f5e0c0b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:36.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.412+0000 7f5e111dd640 1 --2- 172.21.15.131:0/977847993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e0c15aa40 0x7f5e0c0c82c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00c3c0 tx=0x7f5dfc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:36.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.412+0000 7f5e027fc640 1 -- 172.21.15.131:0/977847993 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc0052d0 con 0x7f5e0c15aa40 2023-09-12T14:55:36.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.412+0000 7f5e027fc640 1 -- 172.21.15.131:0/977847993 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5dfc003600 con 0x7f5e0c15aa40 2023-09-12T14:55:36.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.412+0000 7f5e13c42640 1 -- 172.21.15.131:0/977847993 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e0c0c7ac0 con 0x7f5e0c15aa40 2023-09-12T14:55:36.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.412+0000 7f5e027fc640 1 -- 172.21.15.131:0/977847993 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc003a10 con 0x7f5e0c15aa40 2023-09-12T14:55:36.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.412+0000 7f5e13c42640 1 -- 172.21.15.131:0/977847993 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e0c0c6eb0 con 0x7f5e0c15aa40 2023-09-12T14:55:36.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.412+0000 7f5e13c42640 1 -- 172.21.15.131:0/977847993 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e0c073580 con 0x7f5e0c15aa40 2023-09-12T14:55:36.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.416+0000 7f5e027fc640 1 -- 172.21.15.131:0/977847993 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f5dfc003b70 con 0x7f5e0c15aa40 2023-09-12T14:55:36.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.416+0000 7f5e027fc640 1 --2- 172.21.15.131:0/977847993 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5df0065b50 0x7f5df0068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:36.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.416+0000 7f5e027fc640 1 -- 172.21.15.131:0/977847993 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5dfc0829b0 con 0x7f5e0c15aa40 2023-09-12T14:55:36.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.416+0000 7f5e119de640 1 --2- 172.21.15.131:0/977847993 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5df0065b50 0x7f5df0068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:36.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.416+0000 7f5e119de640 1 --2- 172.21.15.131:0/977847993 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5df0065b50 0x7f5df0068010 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5e08001940 tx=0x7f5e08001c10 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:36.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.420+0000 7f5e027fc640 1 -- 172.21.15.131:0/977847993 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f5dfc050300 con 0x7f5e0c15aa40 2023-09-12T14:55:36.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.568+0000 7f5e13c42640 1 -- 172.21.15.131:0/977847993 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5e0c0ade50 con 0x7f5e0c15aa40 2023-09-12T14:55:36.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.572+0000 7f5e027fc640 1 -- 172.21.15.131:0/977847993 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5dfc001070 con 0x7f5e0c15aa40 2023-09-12T14:55:36.574 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-09-12T14:55:36.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.572+0000 7f5e13c42640 1 -- 172.21.15.131:0/977847993 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5df0065b50 msgr2=0x7f5df0068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:36.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.572+0000 7f5e13c42640 1 --2- 172.21.15.131:0/977847993 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5df0065b50 0x7f5df0068010 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5e08001940 tx=0x7f5e08001c10 comp rx=0 tx=0).stop 2023-09-12T14:55:36.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.576+0000 7f5e13c42640 1 -- 172.21.15.131:0/977847993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e0c15aa40 msgr2=0x7f5e0c0c82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:36.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.576+0000 7f5e13c42640 1 --2- 172.21.15.131:0/977847993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e0c15aa40 0x7f5e0c0c82c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00c3c0 tx=0x7f5dfc00c880 comp rx=0 tx=0).stop 2023-09-12T14:55:36.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.576+0000 7f5e13c42640 1 -- 172.21.15.131:0/977847993 shutdown_connections 2023-09-12T14:55:36.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.576+0000 7f5e13c42640 1 --2- 172.21.15.131:0/977847993 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5df0065b50 0x7f5df0068010 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:36.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.576+0000 7f5e13c42640 1 --2- 172.21.15.131:0/977847993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e0c147430 0x7f5e0c0b1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:36.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.576+0000 7f5e13c42640 1 --2- 172.21.15.131:0/977847993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e0c15aa40 0x7f5e0c0c82c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:36.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.576+0000 7f5e13c42640 1 --2- 172.21.15.131:0/977847993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5df8005190 0x7f5e0c0b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:36.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.576+0000 7f5e13c42640 1 -- 172.21.15.131:0/977847993 >> 172.21.15.131:0/977847993 conn(0x7f5e0c098040 msgr2=0x7f5e0c0ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:36.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.576+0000 7f5e13c42640 1 -- 172.21.15.131:0/977847993 shutdown_connections 2023-09-12T14:55:36.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.576+0000 7f5e13c42640 1 -- 172.21.15.131:0/977847993 wait complete. 2023-09-12T14:55:36.595 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T14:55:36.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8846231640 1 -- 172.21.15.131:0/3801897562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88401572c0 msgr2=0x7f88400ac050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:36.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8846231640 1 --2- 172.21.15.131:0/3801897562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88401572c0 0x7f88400ac050 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f884006cf90 tx=0x7f883001c7d0 comp rx=0 tx=0).stop 2023-09-12T14:55:36.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8846231640 1 -- 172.21.15.131:0/3801897562 shutdown_connections 2023-09-12T14:55:36.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8846231640 1 --2- 172.21.15.131:0/3801897562 >> v2:172.21.15.187:3300/0 conn(0x7f884006f3b0 0x7f88400cad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:36.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8846231640 1 --2- 172.21.15.131:0/3801897562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88401572c0 0x7f88400ac050 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:36.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8846231640 1 -- 172.21.15.131:0/3801897562 >> 172.21.15.131:0/3801897562 conn(0x7f8840069840 msgr2=0x7f88400cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:36.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8846231640 1 -- 172.21.15.131:0/3801897562 shutdown_connections 2023-09-12T14:55:36.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8846231640 1 -- 172.21.15.131:0/3801897562 wait complete. 2023-09-12T14:55:36.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8846231640 1 Processor -- start 2023-09-12T14:55:36.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8846231640 1 -- start start 2023-09-12T14:55:36.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8846231640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f884006f3b0 0x7f8840138590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:36.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8846231640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88400abb00 0x7f88401589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:36.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8846231640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88401572c0 0x7f8840158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:36.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8846231640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8840138ad0 con 0x7f88400abb00 2023-09-12T14:55:36.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8846231640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88400a7550 con 0x7f884006f3b0 2023-09-12T14:55:36.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8846231640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88400a7270 con 0x7f88401572c0 2023-09-12T14:55:36.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8836ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88400abb00 0x7f88401589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:36.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8836ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88400abb00 0x7f88401589f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60150/0 (socket says 172.21.15.131:60150) 2023-09-12T14:55:36.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8836ffd640 1 -- 172.21.15.131:0/4243133005 learned_addr learned my addr 172.21.15.131:0/4243133005 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:36.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f883f7fe640 1 --2- 172.21.15.131:0/4243133005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f884006f3b0 0x7f8840138590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:36.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f883ffff640 1 --2- 172.21.15.131:0/4243133005 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88401572c0 0x7f8840158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:36.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8836ffd640 1 -- 172.21.15.131:0/4243133005 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88401572c0 msgr2=0x7f8840158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:36.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8836ffd640 1 --2- 172.21.15.131:0/4243133005 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88401572c0 0x7f8840158f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:36.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8836ffd640 1 -- 172.21.15.131:0/4243133005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f884006f3b0 msgr2=0x7f8840138590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:36.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8836ffd640 1 --2- 172.21.15.131:0/4243133005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f884006f3b0 0x7f8840138590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:36.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8836ffd640 1 -- 172.21.15.131:0/4243133005 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8830009560 con 0x7f88400abb00 2023-09-12T14:55:36.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f883f7fe640 1 --2- 172.21.15.131:0/4243133005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f884006f3b0 0x7f8840138590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:36.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8836ffd640 1 --2- 172.21.15.131:0/4243133005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88400abb00 0x7f88401589f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f882800c490 tx=0x7f8828001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:36.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f883cff9640 1 -- 172.21.15.131:0/4243133005 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8828003600 con 0x7f88400abb00 2023-09-12T14:55:36.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f883cff9640 1 -- 172.21.15.131:0/4243133005 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8828003780 con 0x7f88400abb00 2023-09-12T14:55:36.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8846231640 1 -- 172.21.15.131:0/4243133005 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8840138eb0 con 0x7f88400abb00 2023-09-12T14:55:36.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f883cff9640 1 -- 172.21.15.131:0/4243133005 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8828003bd0 con 0x7f88400abb00 2023-09-12T14:55:36.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8846231640 1 -- 172.21.15.131:0/4243133005 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8840139470 con 0x7f88400abb00 2023-09-12T14:55:36.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.716+0000 7f8846231640 1 -- 172.21.15.131:0/4243133005 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8840139690 con 0x7f88400abb00 2023-09-12T14:55:36.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.720+0000 7f883cff9640 1 -- 172.21.15.131:0/4243133005 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f8828003d30 con 0x7f88400abb00 2023-09-12T14:55:36.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.720+0000 7f883cff9640 1 --2- 172.21.15.131:0/4243133005 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8820065b20 0x7f8820067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:36.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.720+0000 7f883cff9640 1 -- 172.21.15.131:0/4243133005 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8828082a00 con 0x7f88400abb00 2023-09-12T14:55:36.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.720+0000 7f883f7fe640 1 --2- 172.21.15.131:0/4243133005 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8820065b20 0x7f8820067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:36.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.720+0000 7f883f7fe640 1 --2- 172.21.15.131:0/4243133005 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8820065b20 0x7f8820067fe0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f88300af000 tx=0x7f883001b010 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:36.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.724+0000 7f883cff9640 1 -- 172.21.15.131:0/4243133005 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f88280503e0 con 0x7f88400abb00 2023-09-12T14:55:36.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.864+0000 7f8846231640 1 -- 172.21.15.131:0/4243133005 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8840073580 con 0x7f88400abb00 2023-09-12T14:55:36.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.864+0000 7f883cff9640 1 -- 172.21.15.131:0/4243133005 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f88280170a0 con 0x7f88400abb00 2023-09-12T14:55:36.867 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-09-12T14:55:36.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.864+0000 7f8846231640 1 -- 172.21.15.131:0/4243133005 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8820065b20 msgr2=0x7f8820067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:36.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.864+0000 7f8846231640 1 --2- 172.21.15.131:0/4243133005 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8820065b20 0x7f8820067fe0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f88300af000 tx=0x7f883001b010 comp rx=0 tx=0).stop 2023-09-12T14:55:36.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.864+0000 7f8846231640 1 -- 172.21.15.131:0/4243133005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88400abb00 msgr2=0x7f88401589f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:36.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.864+0000 7f8846231640 1 --2- 172.21.15.131:0/4243133005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88400abb00 0x7f88401589f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f882800c490 tx=0x7f8828001e80 comp rx=0 tx=0).stop 2023-09-12T14:55:36.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.864+0000 7f8846231640 1 -- 172.21.15.131:0/4243133005 shutdown_connections 2023-09-12T14:55:36.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.864+0000 7f8846231640 1 --2- 172.21.15.131:0/4243133005 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8820065b20 0x7f8820067fe0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:36.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.864+0000 7f8846231640 1 --2- 172.21.15.131:0/4243133005 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88401572c0 0x7f8840158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:36.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.864+0000 7f8846231640 1 --2- 172.21.15.131:0/4243133005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88400abb00 0x7f88401589f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:36.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.864+0000 7f8846231640 1 --2- 172.21.15.131:0/4243133005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f884006f3b0 0x7f8840138590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:36.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.864+0000 7f8846231640 1 -- 172.21.15.131:0/4243133005 >> 172.21.15.131:0/4243133005 conn(0x7f8840069840 msgr2=0x7f88400cd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:36.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.864+0000 7f8846231640 1 -- 172.21.15.131:0/4243133005 shutdown_connections 2023-09-12T14:55:36.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:36.864+0000 7f8846231640 1 -- 172.21.15.131:0/4243133005 wait complete. 2023-09-12T14:55:36.885 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-12T14:55:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.004+0000 7f4d74f43640 1 -- 172.21.15.131:0/1112883717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d58003980 msgr2=0x7f4d58003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.004+0000 7f4d74f43640 1 --2- 172.21.15.131:0/1112883717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d58003980 0x7f4d58003da0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4d60009b60 tx=0x7f4d6001d260 comp rx=0 tx=0).stop 2023-09-12T14:55:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.004+0000 7f4d6ed9d640 1 -- 172.21.15.131:0/1112883717 reap_dead start 2023-09-12T14:55:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.004+0000 7f4d74f43640 1 -- 172.21.15.131:0/1112883717 shutdown_connections 2023-09-12T14:55:37.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.004+0000 7f4d74f43640 1 -- 172.21.15.131:0/1112883717 >> 172.21.15.131:0/1112883717 conn(0x7f4d70069840 msgr2=0x7f4d700cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:37.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.004+0000 7f4d74f43640 1 -- 172.21.15.131:0/1112883717 shutdown_connections 2023-09-12T14:55:37.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.004+0000 7f4d74f43640 1 -- 172.21.15.131:0/1112883717 wait complete. 2023-09-12T14:55:37.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.008+0000 7f4d74f43640 1 Processor -- start 2023-09-12T14:55:37.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.008+0000 7f4d74f43640 1 -- start start 2023-09-12T14:55:37.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.008+0000 7f4d74f43640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d58003980 0x7f4d70145ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:37.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.008+0000 7f4d74f43640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d70158880 0x7f4d700a64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:37.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.008+0000 7f4d74f43640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d700ae4d0 0x7f4d7015a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:37.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.008+0000 7f4d74f43640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d700ad430 con 0x7f4d700ae4d0 2023-09-12T14:55:37.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.008+0000 7f4d74f43640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d700ad710 con 0x7f4d70158880 2023-09-12T14:55:37.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.008+0000 7f4d74f43640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d700ad9f0 con 0x7f4d58003980 2023-09-12T14:55:37.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.008+0000 7f4d6ed9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d700ae4d0 0x7f4d7015a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:37.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.008+0000 7f4d6ed9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d700ae4d0 0x7f4d7015a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60162/0 (socket says 172.21.15.131:60162) 2023-09-12T14:55:37.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.008+0000 7f4d6ed9d640 1 -- 172.21.15.131:0/3755827090 learned_addr learned my addr 172.21.15.131:0/3755827090 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:37.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.008+0000 7f4d6ed9d640 1 -- 172.21.15.131:0/3755827090 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d58003980 msgr2=0x7f4d70145ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:37.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.008+0000 7f4d6ed9d640 1 --2- 172.21.15.131:0/3755827090 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d58003980 0x7f4d70145ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:37.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.008+0000 7f4d6ed9d640 1 -- 172.21.15.131:0/3755827090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d70158880 msgr2=0x7f4d700a64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:37.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.008+0000 7f4d6dd9b640 1 --2- 172.21.15.131:0/3755827090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d70158880 0x7f4d700a64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:37.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.008+0000 7f4d6ed9d640 1 --2- 172.21.15.131:0/3755827090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d70158880 0x7f4d700a64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:37.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.008+0000 7f4d6ed9d640 1 -- 172.21.15.131:0/3755827090 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d700adcd0 con 0x7f4d700ae4d0 2023-09-12T14:55:37.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.008+0000 7f4d6ed9d640 1 --2- 172.21.15.131:0/3755827090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d700ae4d0 0x7f4d7015a570 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4d70070fa0 tx=0x7f4d6800dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:37.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.008+0000 7f4d5effd640 1 -- 172.21.15.131:0/3755827090 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d68015030 con 0x7f4d700ae4d0 2023-09-12T14:55:37.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.008+0000 7f4d5effd640 1 -- 172.21.15.131:0/3755827090 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4d68003e80 con 0x7f4d700ae4d0 2023-09-12T14:55:37.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.008+0000 7f4d74f43640 1 -- 172.21.15.131:0/3755827090 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d700adfb0 con 0x7f4d700ae4d0 2023-09-12T14:55:37.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.008+0000 7f4d5effd640 1 -- 172.21.15.131:0/3755827090 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6800ba40 con 0x7f4d700ae4d0 2023-09-12T14:55:37.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.008+0000 7f4d74f43640 1 -- 172.21.15.131:0/3755827090 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d700a6a10 con 0x7f4d700ae4d0 2023-09-12T14:55:37.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.008+0000 7f4d74f43640 1 -- 172.21.15.131:0/3755827090 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d700ae1a0 con 0x7f4d700ae4d0 2023-09-12T14:55:37.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.008+0000 7f4d5effd640 1 -- 172.21.15.131:0/3755827090 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f4d68018070 con 0x7f4d700ae4d0 2023-09-12T14:55:37.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.008+0000 7f4d5effd640 1 --2- 172.21.15.131:0/3755827090 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4d50065b50 0x7f4d50068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:37.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.008+0000 7f4d5effd640 1 -- 172.21.15.131:0/3755827090 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d68016070 con 0x7f4d700ae4d0 2023-09-12T14:55:37.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.012+0000 7f4d6e59c640 1 --2- 172.21.15.131:0/3755827090 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4d50065b50 0x7f4d50068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:37.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.012+0000 7f4d6e59c640 1 --2- 172.21.15.131:0/3755827090 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4d50065b50 0x7f4d50068010 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4d60009b60 tx=0x7f4d58005800 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:37.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.016+0000 7f4d5effd640 1 -- 172.21.15.131:0/3755827090 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f4d68050e90 con 0x7f4d700ae4d0 2023-09-12T14:55:37.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.176+0000 7f4d74f43640 1 -- 172.21.15.131:0/3755827090 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4d70073580 con 0x7f4d700ae4d0 2023-09-12T14:55:37.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.180+0000 7f4d5effd640 1 -- 172.21.15.131:0/3755827090 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4d680544b0 con 0x7f4d700ae4d0 2023-09-12T14:55:37.182 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2023-09-12T14:55:37.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.180+0000 7f4d74f43640 1 -- 172.21.15.131:0/3755827090 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4d50065b50 msgr2=0x7f4d50068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:37.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.180+0000 7f4d74f43640 1 --2- 172.21.15.131:0/3755827090 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4d50065b50 0x7f4d50068010 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4d60009b60 tx=0x7f4d58005800 comp rx=0 tx=0).stop 2023-09-12T14:55:37.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.180+0000 7f4d74f43640 1 -- 172.21.15.131:0/3755827090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d700ae4d0 msgr2=0x7f4d7015a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:37.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.180+0000 7f4d74f43640 1 --2- 172.21.15.131:0/3755827090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d700ae4d0 0x7f4d7015a570 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4d70070fa0 tx=0x7f4d6800dd80 comp rx=0 tx=0).stop 2023-09-12T14:55:37.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.180+0000 7f4d74f43640 1 -- 172.21.15.131:0/3755827090 shutdown_connections 2023-09-12T14:55:37.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.180+0000 7f4d74f43640 1 --2- 172.21.15.131:0/3755827090 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4d50065b50 0x7f4d50068010 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:37.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.180+0000 7f4d74f43640 1 --2- 172.21.15.131:0/3755827090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d700ae4d0 0x7f4d7015a570 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:37.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.180+0000 7f4d74f43640 1 --2- 172.21.15.131:0/3755827090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d70158880 0x7f4d700a64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:37.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.180+0000 7f4d74f43640 1 --2- 172.21.15.131:0/3755827090 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d58003980 0x7f4d70145ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:37.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.180+0000 7f4d74f43640 1 -- 172.21.15.131:0/3755827090 >> 172.21.15.131:0/3755827090 conn(0x7f4d70069840 msgr2=0x7f4d70152230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:37.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.180+0000 7f4d74f43640 1 -- 172.21.15.131:0/3755827090 shutdown_connections 2023-09-12T14:55:37.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.180+0000 7f4d74f43640 1 -- 172.21.15.131:0/3755827090 wait complete. 2023-09-12T14:55:37.199 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi187.front.sepia.ceph.com... 2023-09-12T14:55:37.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:37.200 DEBUG:teuthology.orchestra.run.smithi187:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-09-12T14:55:37.297 INFO:teuthology.orchestra.run:waiting for 300 2023-09-12T14:55:37.298 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2023-09-12T14:55:37.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:37.298 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-09-12T14:55:37.314 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:55:37.315 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-12T14:55:37.321 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:55:37.321 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-12T14:55:37.371 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:55:37.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.536+0000 7f6d79c5a640 1 -- 172.21.15.131:0/2348300736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d740c9370 msgr2=0x7f6d740c9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:37.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.536+0000 7f6d79c5a640 1 --2- 172.21.15.131:0/2348300736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d740c9370 0x7f6d740c9780 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6d740cd200 tx=0x7f6d6c01ccf0 comp rx=0 tx=0).stop 2023-09-12T14:55:37.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.536+0000 7f6d79c5a640 1 -- 172.21.15.131:0/2348300736 shutdown_connections 2023-09-12T14:55:37.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.536+0000 7f6d79c5a640 1 --2- 172.21.15.131:0/2348300736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d740c9370 0x7f6d740c9780 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:37.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.536+0000 7f6d79c5a640 1 --2- 172.21.15.131:0/2348300736 >> v2:172.21.15.187:3300/0 conn(0x7f6d74156930 0x7f6d74159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:37.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.536+0000 7f6d79c5a640 1 -- 172.21.15.131:0/2348300736 >> 172.21.15.131:0/2348300736 conn(0x7f6d74069840 msgr2=0x7f6d740cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:37.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.536+0000 7f6d79c5a640 1 -- 172.21.15.131:0/2348300736 shutdown_connections 2023-09-12T14:55:37.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.536+0000 7f6d79c5a640 1 -- 172.21.15.131:0/2348300736 wait complete. 2023-09-12T14:55:37.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.536+0000 7f6d79c5a640 1 Processor -- start 2023-09-12T14:55:37.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.536+0000 7f6d79c5a640 1 -- start start 2023-09-12T14:55:37.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d79c5a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d740abb40 0x7f6d74149990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:37.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d79c5a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d740c9370 0x7f6d74070840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:37.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d79c5a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d74156930 0x7f6d740aab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:37.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d79c5a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d740ad3c0 con 0x7f6d740abb40 2023-09-12T14:55:37.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d79c5a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7406f0a0 con 0x7f6d740c9370 2023-09-12T14:55:37.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d79c5a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d740ac840 con 0x7f6d74156930 2023-09-12T14:55:37.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d737fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d740abb40 0x7f6d74149990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:37.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d737fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d740abb40 0x7f6d74149990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60182/0 (socket says 172.21.15.131:60182) 2023-09-12T14:55:37.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d737fe640 1 -- 172.21.15.131:0/2081550959 learned_addr learned my addr 172.21.15.131:0/2081550959 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:37.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d72ffd640 1 --2- 172.21.15.131:0/2081550959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d740c9370 0x7f6d74070840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:37.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d73fff640 1 --2- 172.21.15.131:0/2081550959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d74156930 0x7f6d740aab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:37.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d737fe640 1 -- 172.21.15.131:0/2081550959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d74156930 msgr2=0x7f6d740aab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:37.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d737fe640 1 --2- 172.21.15.131:0/2081550959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d74156930 0x7f6d740aab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:37.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d737fe640 1 -- 172.21.15.131:0/2081550959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d740c9370 msgr2=0x7f6d74070840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:37.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d737fe640 1 --2- 172.21.15.131:0/2081550959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d740c9370 0x7f6d74070840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:37.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d737fe640 1 -- 172.21.15.131:0/2081550959 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d6c0098e0 con 0x7f6d740abb40 2023-09-12T14:55:37.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d72ffd640 1 --2- 172.21.15.131:0/2081550959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d740c9370 0x7f6d74070840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:55:37.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d737fe640 1 --2- 172.21.15.131:0/2081550959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d740abb40 0x7f6d74149990 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400c940 tx=0x7f6d6400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:37.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d5bfff640 1 -- 172.21.15.131:0/2081550959 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64004220 con 0x7f6d740abb40 2023-09-12T14:55:37.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d5bfff640 1 -- 172.21.15.131:0/2081550959 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6d64004470 con 0x7f6d740abb40 2023-09-12T14:55:37.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d5bfff640 1 -- 172.21.15.131:0/2081550959 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64007c10 con 0x7f6d740abb40 2023-09-12T14:55:37.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d79c5a640 1 -- 172.21.15.131:0/2081550959 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d740acb20 con 0x7f6d740abb40 2023-09-12T14:55:37.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d79c5a640 1 -- 172.21.15.131:0/2081550959 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d740ad0e0 con 0x7f6d740abb40 2023-09-12T14:55:37.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d79c5a640 1 -- 172.21.15.131:0/2081550959 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d7406f260 con 0x7f6d740abb40 2023-09-12T14:55:37.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d5bfff640 1 -- 172.21.15.131:0/2081550959 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f6d64003600 con 0x7f6d740abb40 2023-09-12T14:55:37.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d5bfff640 1 --2- 172.21.15.131:0/2081550959 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6d60065b20 0x7f6d60067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:37.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d72ffd640 1 --2- 172.21.15.131:0/2081550959 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6d60065b20 0x7f6d60067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:37.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.540+0000 7f6d5bfff640 1 -- 172.21.15.131:0/2081550959 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d64083710 con 0x7f6d740abb40 2023-09-12T14:55:37.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.544+0000 7f6d72ffd640 1 --2- 172.21.15.131:0/2081550959 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6d60065b20 0x7f6d60067fe0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6d74149ed0 tx=0x7f6d68009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:37.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.544+0000 7f6d5bfff640 1 -- 172.21.15.131:0/2081550959 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f6d7406f260 con 0x7f6d740abb40 2023-09-12T14:55:37.696 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.692+0000 7f6d79c5a640 1 -- 172.21.15.131:0/2081550959 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d7406f260 con 0x7f6d740abb40 2023-09-12T14:55:37.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.692+0000 7f6d5bfff640 1 -- 172.21.15.131:0/2081550959 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f6d7406f260 con 0x7f6d740abb40 2023-09-12T14:55:37.697 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:37.697 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:54:59.694778+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:54:55.689983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T14:54:57.029447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:55:37.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.696+0000 7f6d79c5a640 1 -- 172.21.15.131:0/2081550959 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6d60065b20 msgr2=0x7f6d60067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:37.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.696+0000 7f6d79c5a640 1 --2- 172.21.15.131:0/2081550959 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6d60065b20 0x7f6d60067fe0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6d74149ed0 tx=0x7f6d68009040 comp rx=0 tx=0).stop 2023-09-12T14:55:37.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.696+0000 7f6d79c5a640 1 -- 172.21.15.131:0/2081550959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d740abb40 msgr2=0x7f6d74149990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:37.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.696+0000 7f6d79c5a640 1 --2- 172.21.15.131:0/2081550959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d740abb40 0x7f6d74149990 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400c940 tx=0x7f6d6400ce00 comp rx=0 tx=0).stop 2023-09-12T14:55:37.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.696+0000 7f6d79c5a640 1 -- 172.21.15.131:0/2081550959 shutdown_connections 2023-09-12T14:55:37.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.696+0000 7f6d79c5a640 1 --2- 172.21.15.131:0/2081550959 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6d60065b20 0x7f6d60067fe0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:37.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.696+0000 7f6d79c5a640 1 --2- 172.21.15.131:0/2081550959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d74156930 0x7f6d740aab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:37.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.696+0000 7f6d79c5a640 1 --2- 172.21.15.131:0/2081550959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d740c9370 0x7f6d74070840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:37.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.696+0000 7f6d79c5a640 1 --2- 172.21.15.131:0/2081550959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d740abb40 0x7f6d74149990 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:37.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.696+0000 7f6d79c5a640 1 -- 172.21.15.131:0/2081550959 >> 172.21.15.131:0/2081550959 conn(0x7f6d74069840 msgr2=0x7f6d7414b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:37.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.696+0000 7f6d79c5a640 1 -- 172.21.15.131:0/2081550959 shutdown_connections 2023-09-12T14:55:37.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.696+0000 7f6d79c5a640 1 -- 172.21.15.131:0/2081550959 wait complete. 2023-09-12T14:55:37.716 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T14:54:45.654004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T14:54:55.689983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T14:54:57.029447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-12T14:55:37.717 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T14:55:37.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc8d5c0640 1 -- 172.21.15.131:0/1298156070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc880b2100 msgr2=0x7fdc8806be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:37.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc8d5c0640 1 --2- 172.21.15.131:0/1298156070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc880b2100 0x7fdc8806be70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdc70009840 tx=0x7fdc7001c7d0 comp rx=0 tx=0).stop 2023-09-12T14:55:37.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc8d5c0640 1 -- 172.21.15.131:0/1298156070 shutdown_connections 2023-09-12T14:55:37.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc8d5c0640 1 --2- 172.21.15.131:0/1298156070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc880b2100 0x7fdc8806be70 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:37.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc8d5c0640 1 -- 172.21.15.131:0/1298156070 >> 172.21.15.131:0/1298156070 conn(0x7fdc8806a950 msgr2=0x7fdc880af5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:37.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc8d5c0640 1 -- 172.21.15.131:0/1298156070 shutdown_connections 2023-09-12T14:55:37.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc8d5c0640 1 -- 172.21.15.131:0/1298156070 wait complete. 2023-09-12T14:55:37.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc8d5c0640 1 Processor -- start 2023-09-12T14:55:37.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc8d5c0640 1 -- start start 2023-09-12T14:55:37.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc8d5c0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc8806c3b0 0x7fdc8815a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:37.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc8d5c0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc880b2100 0x7fdc88157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:37.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc8d5c0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc881556e0 0x7fdc88151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:37.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc8d5c0640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8815b920 con 0x7fdc8806c3b0 2023-09-12T14:55:37.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc8d5c0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8815bd40 con 0x7fdc881556e0 2023-09-12T14:55:37.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc8d5c0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8815aca0 con 0x7fdc880b2100 2023-09-12T14:55:37.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc86ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc8806c3b0 0x7fdc8815a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:37.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc86ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc8806c3b0 0x7fdc8815a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60210/0 (socket says 172.21.15.131:60210) 2023-09-12T14:55:37.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc86ffd640 1 -- 172.21.15.131:0/2047225362 learned_addr learned my addr 172.21.15.131:0/2047225362 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:37.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc877fe640 1 --2- 172.21.15.131:0/2047225362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc881556e0 0x7fdc88151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:37.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc86ffd640 1 -- 172.21.15.131:0/2047225362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc880b2100 msgr2=0x7fdc88157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:37.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc86ffd640 1 --2- 172.21.15.131:0/2047225362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc880b2100 0x7fdc88157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:37.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc86ffd640 1 -- 172.21.15.131:0/2047225362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc881556e0 msgr2=0x7fdc88151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:37.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc86ffd640 1 --2- 172.21.15.131:0/2047225362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc881556e0 0x7fdc88151800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:37.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc86ffd640 1 -- 172.21.15.131:0/2047225362 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc700094a0 con 0x7fdc8806c3b0 2023-09-12T14:55:37.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc877fe640 1 --2- 172.21.15.131:0/2047225362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc881556e0 0x7fdc88151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:37.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc86ffd640 1 --2- 172.21.15.131:0/2047225362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc8806c3b0 0x7fdc8815a330 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000cb50 tx=0x7fdc8000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:37.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc7f7fe640 1 -- 172.21.15.131:0/2047225362 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8000aa50 con 0x7fdc8806c3b0 2023-09-12T14:55:37.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc7f7fe640 1 -- 172.21.15.131:0/2047225362 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdc8000ce90 con 0x7fdc8806c3b0 2023-09-12T14:55:37.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc8d5c0640 1 -- 172.21.15.131:0/2047225362 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc8815af80 con 0x7fdc8806c3b0 2023-09-12T14:55:37.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc7f7fe640 1 -- 172.21.15.131:0/2047225362 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc80012420 con 0x7fdc8806c3b0 2023-09-12T14:55:37.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.828+0000 7fdc8d5c0640 1 -- 172.21.15.131:0/2047225362 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc8815b540 con 0x7fdc8806c3b0 2023-09-12T14:55:37.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.832+0000 7fdc8d5c0640 1 -- 172.21.15.131:0/2047225362 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc8815ba60 con 0x7fdc8806c3b0 2023-09-12T14:55:37.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.832+0000 7fdc7f7fe640 1 -- 172.21.15.131:0/2047225362 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fdc8001a3b0 con 0x7fdc8806c3b0 2023-09-12T14:55:37.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.832+0000 7fdc7f7fe640 1 --2- 172.21.15.131:0/2047225362 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdc68065b20 0x7fdc68067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:37.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.832+0000 7fdc867fc640 1 --2- 172.21.15.131:0/2047225362 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdc68065b20 0x7fdc68067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:37.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.832+0000 7fdc7f7fe640 1 -- 172.21.15.131:0/2047225362 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc800836b0 con 0x7fdc8806c3b0 2023-09-12T14:55:37.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.832+0000 7fdc867fc640 1 --2- 172.21.15.131:0/2047225362 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdc68065b20 0x7fdc68067fe0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdc700af000 tx=0x7fdc70019570 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:37.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.836+0000 7fdc7f7fe640 1 -- 172.21.15.131:0/2047225362 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fdc8815ba60 con 0x7fdc8806c3b0 2023-09-12T14:55:37.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.988+0000 7fdc8d5c0640 1 -- 172.21.15.131:0/2047225362 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdc8815ba60 con 0x7fdc8806c3b0 2023-09-12T14:55:37.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.988+0000 7fdc7f7fe640 1 -- 172.21.15.131:0/2047225362 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdc8815ba60 con 0x7fdc8806c3b0 2023-09-12T14:55:37.992 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-09-12T14:55:37.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.992+0000 7fdc8d5c0640 1 -- 172.21.15.131:0/2047225362 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdc68065b20 msgr2=0x7fdc68067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:37.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.992+0000 7fdc8d5c0640 1 --2- 172.21.15.131:0/2047225362 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdc68065b20 0x7fdc68067fe0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdc700af000 tx=0x7fdc70019570 comp rx=0 tx=0).stop 2023-09-12T14:55:37.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.992+0000 7fdc8d5c0640 1 -- 172.21.15.131:0/2047225362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc8806c3b0 msgr2=0x7fdc8815a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:37.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.992+0000 7fdc8d5c0640 1 --2- 172.21.15.131:0/2047225362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc8806c3b0 0x7fdc8815a330 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000cb50 tx=0x7fdc8000cc40 comp rx=0 tx=0).stop 2023-09-12T14:55:37.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.992+0000 7fdc8d5c0640 1 -- 172.21.15.131:0/2047225362 shutdown_connections 2023-09-12T14:55:37.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.992+0000 7fdc8d5c0640 1 --2- 172.21.15.131:0/2047225362 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdc68065b20 0x7fdc68067fe0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:37.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.992+0000 7fdc8d5c0640 1 --2- 172.21.15.131:0/2047225362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc880b2100 0x7fdc88157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:37.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.992+0000 7fdc8d5c0640 1 --2- 172.21.15.131:0/2047225362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc881556e0 0x7fdc88151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:37.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.992+0000 7fdc8d5c0640 1 --2- 172.21.15.131:0/2047225362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc8806c3b0 0x7fdc8815a330 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:37.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.992+0000 7fdc8d5c0640 1 -- 172.21.15.131:0/2047225362 >> 172.21.15.131:0/2047225362 conn(0x7fdc8806a950 msgr2=0x7fdc88162f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:37.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.992+0000 7fdc8d5c0640 1 -- 172.21.15.131:0/2047225362 shutdown_connections 2023-09-12T14:55:37.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:37.992+0000 7fdc8d5c0640 1 -- 172.21.15.131:0/2047225362 wait complete. 2023-09-12T14:55:38.011 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T14:55:38.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.124+0000 7fdaf55fc640 1 -- 172.21.15.131:0/1796139238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaf015d570 msgr2=0x7fdaf00cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:38.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.124+0000 7fdaf55fc640 1 --2- 172.21.15.131:0/1796139238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaf015d570 0x7fdaf00cb5d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00ceb40 tx=0x7fdae801cc00 comp rx=0 tx=0).stop 2023-09-12T14:55:38.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.124+0000 7fdaf55fc640 1 -- 172.21.15.131:0/1796139238 shutdown_connections 2023-09-12T14:55:38.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.124+0000 7fdaf55fc640 1 --2- 172.21.15.131:0/1796139238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdaf00b1ff0 0x7fdaf00b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:38.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.124+0000 7fdaf55fc640 1 --2- 172.21.15.131:0/1796139238 >> v2:172.21.15.187:3300/0 conn(0x7fdaf00cf700 0x7fdaf00cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:38.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.124+0000 7fdaf55fc640 1 --2- 172.21.15.131:0/1796139238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaf015d570 0x7fdaf00cb5d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:38.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.124+0000 7fdaf55fc640 1 -- 172.21.15.131:0/1796139238 >> 172.21.15.131:0/1796139238 conn(0x7fdaf00cebe0 msgr2=0x7fdaf00ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:38.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.124+0000 7fdaf55fc640 1 -- 172.21.15.131:0/1796139238 shutdown_connections 2023-09-12T14:55:38.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.124+0000 7fdaf55fc640 1 -- 172.21.15.131:0/1796139238 wait complete. 2023-09-12T14:55:38.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.124+0000 7fdaf55fc640 1 Processor -- start 2023-09-12T14:55:38.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.124+0000 7fdaf55fc640 1 -- start start 2023-09-12T14:55:38.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdaf55fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdaf00b1ff0 0x7fdaf0152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:38.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdaf55fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdaf00cf700 0x7fdaf01f7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:38.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdaf55fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaf015d570 0x7fdaf01f7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:38.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdaf55fc640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf0162e20 con 0x7fdaf015d570 2023-09-12T14:55:38.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdaf55fc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf0163100 con 0x7fdaf00cf700 2023-09-12T14:55:38.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdaf55fc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf01633e0 con 0x7fdaf00b1ff0 2023-09-12T14:55:38.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdaef7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaf015d570 0x7fdaf01f7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:38.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdaef7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaf015d570 0x7fdaf01f7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60222/0 (socket says 172.21.15.131:60222) 2023-09-12T14:55:38.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdaef7fe640 1 -- 172.21.15.131:0/793099014 learned_addr learned my addr 172.21.15.131:0/793099014 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:38.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdaeeffd640 1 --2- 172.21.15.131:0/793099014 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdaf00b1ff0 0x7fdaf0152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:38.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdaee7fc640 1 --2- 172.21.15.131:0/793099014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdaf00cf700 0x7fdaf01f7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:38.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdaef7fe640 1 -- 172.21.15.131:0/793099014 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdaf00b1ff0 msgr2=0x7fdaf0152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:38.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdaef7fe640 1 --2- 172.21.15.131:0/793099014 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdaf00b1ff0 0x7fdaf0152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:38.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdaef7fe640 1 -- 172.21.15.131:0/793099014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdaf00cf700 msgr2=0x7fdaf01f7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:38.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdaef7fe640 1 --2- 172.21.15.131:0/793099014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdaf00cf700 0x7fdaf01f7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:38.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdaef7fe640 1 -- 172.21.15.131:0/793099014 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae8009880 con 0x7fdaf015d570 2023-09-12T14:55:38.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdaee7fc640 1 --2- 172.21.15.131:0/793099014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdaf00cf700 0x7fdaf01f7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:38.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdaef7fe640 1 --2- 172.21.15.131:0/793099014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaf015d570 0x7fdaf01f7d80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdae400c940 tx=0x7fdae400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:38.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdad77fe640 1 -- 172.21.15.131:0/793099014 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae4004220 con 0x7fdaf015d570 2023-09-12T14:55:38.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdad77fe640 1 -- 172.21.15.131:0/793099014 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdae4004470 con 0x7fdaf015d570 2023-09-12T14:55:38.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdaf55fc640 1 -- 172.21.15.131:0/793099014 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf01636c0 con 0x7fdaf015d570 2023-09-12T14:55:38.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdad77fe640 1 -- 172.21.15.131:0/793099014 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae4007bf0 con 0x7fdaf015d570 2023-09-12T14:55:38.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdaf55fc640 1 -- 172.21.15.131:0/793099014 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf0152c00 con 0x7fdaf015d570 2023-09-12T14:55:38.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdaf55fc640 1 -- 172.21.15.131:0/793099014 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf00cc590 con 0x7fdaf015d570 2023-09-12T14:55:38.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdad77fe640 1 -- 172.21.15.131:0/793099014 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fdae4007d50 con 0x7fdaf015d570 2023-09-12T14:55:38.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdad77fe640 1 --2- 172.21.15.131:0/793099014 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdadc065b20 0x7fdadc067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:38.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdad77fe640 1 -- 172.21.15.131:0/793099014 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fdae40837f0 con 0x7fdaf015d570 2023-09-12T14:55:38.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdaeeffd640 1 --2- 172.21.15.131:0/793099014 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdadc065b20 0x7fdadc067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:38.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.128+0000 7fdaeeffd640 1 --2- 172.21.15.131:0/793099014 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdadc065b20 0x7fdadc067fe0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdae80af000 tx=0x7fdae801b440 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:38.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.132+0000 7fdad77fe640 1 -- 172.21.15.131:0/793099014 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fdae40510c0 con 0x7fdaf015d570 2023-09-12T14:55:38.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.272+0000 7fdaf55fc640 1 -- 172.21.15.131:0/793099014 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdaf0073580 con 0x7fdaf015d570 2023-09-12T14:55:38.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.272+0000 7fdad77fe640 1 -- 172.21.15.131:0/793099014 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdae401d0a0 con 0x7fdaf015d570 2023-09-12T14:55:38.277 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-09-12T14:55:38.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.276+0000 7fdaf55fc640 1 -- 172.21.15.131:0/793099014 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdadc065b20 msgr2=0x7fdadc067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:38.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.276+0000 7fdaf55fc640 1 --2- 172.21.15.131:0/793099014 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdadc065b20 0x7fdadc067fe0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdae80af000 tx=0x7fdae801b440 comp rx=0 tx=0).stop 2023-09-12T14:55:38.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.276+0000 7fdaf55fc640 1 -- 172.21.15.131:0/793099014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaf015d570 msgr2=0x7fdaf01f7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:38.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.276+0000 7fdaf55fc640 1 --2- 172.21.15.131:0/793099014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaf015d570 0x7fdaf01f7d80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdae400c940 tx=0x7fdae400ce00 comp rx=0 tx=0).stop 2023-09-12T14:55:38.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.276+0000 7fdaf55fc640 1 -- 172.21.15.131:0/793099014 shutdown_connections 2023-09-12T14:55:38.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.276+0000 7fdaf55fc640 1 --2- 172.21.15.131:0/793099014 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdadc065b20 0x7fdadc067fe0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:38.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.276+0000 7fdaf55fc640 1 --2- 172.21.15.131:0/793099014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaf015d570 0x7fdaf01f7d80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:38.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.276+0000 7fdaf55fc640 1 --2- 172.21.15.131:0/793099014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdaf00cf700 0x7fdaf01f7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:38.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.276+0000 7fdaf55fc640 1 --2- 172.21.15.131:0/793099014 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdaf00b1ff0 0x7fdaf0152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:38.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.276+0000 7fdaf55fc640 1 -- 172.21.15.131:0/793099014 >> 172.21.15.131:0/793099014 conn(0x7fdaf00cebe0 msgr2=0x7fdaf006e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:38.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.276+0000 7fdaf55fc640 1 -- 172.21.15.131:0/793099014 shutdown_connections 2023-09-12T14:55:38.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.276+0000 7fdaf55fc640 1 -- 172.21.15.131:0/793099014 wait complete. 2023-09-12T14:55:38.295 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T14:55:38.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.408+0000 7fa6f0f50640 1 -- 172.21.15.131:0/2756637056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6d4005190 msgr2=0x7fa6d400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:38.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.408+0000 7fa6f0f50640 1 --2- 172.21.15.131:0/2756637056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6d4005190 0x7fa6d400b990 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4009ee0 tx=0x7fa6e401d610 comp rx=0 tx=0).stop 2023-09-12T14:55:38.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.408+0000 7fa6f0f50640 1 -- 172.21.15.131:0/2756637056 shutdown_connections 2023-09-12T14:55:38.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.408+0000 7fa6f0f50640 1 --2- 172.21.15.131:0/2756637056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6d4005190 0x7fa6d400b990 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:38.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.408+0000 7fa6f0f50640 1 -- 172.21.15.131:0/2756637056 >> 172.21.15.131:0/2756637056 conn(0x7fa6ec069840 msgr2=0x7fa6ec0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:38.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.408+0000 7fa6f0f50640 1 -- 172.21.15.131:0/2756637056 shutdown_connections 2023-09-12T14:55:38.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.408+0000 7fa6f0f50640 1 -- 172.21.15.131:0/2756637056 wait complete. 2023-09-12T14:55:38.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.408+0000 7fa6f0f50640 1 Processor -- start 2023-09-12T14:55:38.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.408+0000 7fa6f0f50640 1 -- start start 2023-09-12T14:55:38.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.408+0000 7fa6f0f50640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6d4005190 0x7fa6ec155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:38.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.408+0000 7fa6f0f50640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6ec155d60 0x7fa6ec0ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:38.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.408+0000 7fa6f0f50640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6ec0cae40 0x7fa6ec0cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:38.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.408+0000 7fa6f0f50640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec13d130 con 0x7fa6ec0cae40 2023-09-12T14:55:38.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.408+0000 7fa6f0f50640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec1561e0 con 0x7fa6d4005190 2023-09-12T14:55:38.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.408+0000 7fa6f0f50640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec13c520 con 0x7fa6ec155d60 2023-09-12T14:55:38.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.408+0000 7fa6ead9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6ec0cae40 0x7fa6ec0cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:38.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.408+0000 7fa6ead9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6ec0cae40 0x7fa6ec0cb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48586/0 (socket says 172.21.15.131:48586) 2023-09-12T14:55:38.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.408+0000 7fa6ead9d640 1 -- 172.21.15.131:0/1158713502 learned_addr learned my addr 172.21.15.131:0/1158713502 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:38.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.412+0000 7fa6e9d9b640 1 --2- 172.21.15.131:0/1158713502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6ec155d60 0x7fa6ec0ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:38.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.412+0000 7fa6ea59c640 1 --2- 172.21.15.131:0/1158713502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6d4005190 0x7fa6ec155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:38.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.412+0000 7fa6ead9d640 1 -- 172.21.15.131:0/1158713502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6ec155d60 msgr2=0x7fa6ec0ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:38.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.412+0000 7fa6ead9d640 1 --2- 172.21.15.131:0/1158713502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6ec155d60 0x7fa6ec0ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:38.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.412+0000 7fa6ead9d640 1 -- 172.21.15.131:0/1158713502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6d4005190 msgr2=0x7fa6ec155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:38.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.412+0000 7fa6ead9d640 1 --2- 172.21.15.131:0/1158713502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6d4005190 0x7fa6ec155820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:38.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.412+0000 7fa6ead9d640 1 -- 172.21.15.131:0/1158713502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec13c800 con 0x7fa6ec0cae40 2023-09-12T14:55:38.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.412+0000 7fa6e9d9b640 1 --2- 172.21.15.131:0/1158713502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6ec155d60 0x7fa6ec0ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:38.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.412+0000 7fa6ead9d640 1 --2- 172.21.15.131:0/1158713502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6ec0cae40 0x7fa6ec0cb2c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa6e401d440 tx=0x7fa6e4004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:38.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.412+0000 7fa6daffd640 1 -- 172.21.15.131:0/1158713502 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4009200 con 0x7fa6ec0cae40 2023-09-12T14:55:38.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.412+0000 7fa6daffd640 1 -- 172.21.15.131:0/1158713502 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa6e4004870 con 0x7fa6ec0cae40 2023-09-12T14:55:38.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.412+0000 7fa6daffd640 1 -- 172.21.15.131:0/1158713502 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4004c90 con 0x7fa6ec0cae40 2023-09-12T14:55:38.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.412+0000 7fa6f0f50640 1 -- 172.21.15.131:0/1158713502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6ec13cae0 con 0x7fa6ec0cae40 2023-09-12T14:55:38.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.412+0000 7fa6f0f50640 1 -- 172.21.15.131:0/1158713502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ec14b870 con 0x7fa6ec0cae40 2023-09-12T14:55:38.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.412+0000 7fa6f0f50640 1 -- 172.21.15.131:0/1158713502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6ec073580 con 0x7fa6ec0cae40 2023-09-12T14:55:38.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.412+0000 7fa6daffd640 1 -- 172.21.15.131:0/1158713502 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fa6e4005030 con 0x7fa6ec0cae40 2023-09-12T14:55:38.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.412+0000 7fa6daffd640 1 --2- 172.21.15.131:0/1158713502 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa6cc065b50 0x7fa6cc068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:38.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.412+0000 7fa6ea59c640 1 --2- 172.21.15.131:0/1158713502 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa6cc065b50 0x7fa6cc068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:38.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.412+0000 7fa6daffd640 1 -- 172.21.15.131:0/1158713502 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa6e411dc80 con 0x7fa6ec0cae40 2023-09-12T14:55:38.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.412+0000 7fa6ea59c640 1 --2- 172.21.15.131:0/1158713502 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa6cc065b50 0x7fa6cc068010 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc0011f0 tx=0x7fa6dc009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:38.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.416+0000 7fa6daffd640 1 -- 172.21.15.131:0/1158713502 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fa6e40eb4c0 con 0x7fa6ec0cae40 2023-09-12T14:55:38.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.556+0000 7fa6f0f50640 1 -- 172.21.15.131:0/1158713502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa6ec0cd4d0 con 0x7fa6ec0cae40 2023-09-12T14:55:38.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.560+0000 7fa6daffd640 1 -- 172.21.15.131:0/1158713502 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa6e40bb030 con 0x7fa6ec0cae40 2023-09-12T14:55:38.562 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-09-12T14:55:38.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.560+0000 7fa6f0f50640 1 -- 172.21.15.131:0/1158713502 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa6cc065b50 msgr2=0x7fa6cc068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:38.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.560+0000 7fa6f0f50640 1 --2- 172.21.15.131:0/1158713502 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa6cc065b50 0x7fa6cc068010 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc0011f0 tx=0x7fa6dc009040 comp rx=0 tx=0).stop 2023-09-12T14:55:38.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.560+0000 7fa6f0f50640 1 -- 172.21.15.131:0/1158713502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6ec0cae40 msgr2=0x7fa6ec0cb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:38.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.560+0000 7fa6f0f50640 1 --2- 172.21.15.131:0/1158713502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6ec0cae40 0x7fa6ec0cb2c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa6e401d440 tx=0x7fa6e4004680 comp rx=0 tx=0).stop 2023-09-12T14:55:38.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.560+0000 7fa6f0f50640 1 -- 172.21.15.131:0/1158713502 shutdown_connections 2023-09-12T14:55:38.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.560+0000 7fa6f0f50640 1 --2- 172.21.15.131:0/1158713502 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa6cc065b50 0x7fa6cc068010 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:38.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.560+0000 7fa6f0f50640 1 --2- 172.21.15.131:0/1158713502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6ec155d60 0x7fa6ec0ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:38.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.560+0000 7fa6f0f50640 1 --2- 172.21.15.131:0/1158713502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6ec0cae40 0x7fa6ec0cb2c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:38.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.560+0000 7fa6f0f50640 1 --2- 172.21.15.131:0/1158713502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6d4005190 0x7fa6ec155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:38.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.560+0000 7fa6f0f50640 1 -- 172.21.15.131:0/1158713502 >> 172.21.15.131:0/1158713502 conn(0x7fa6ec069840 msgr2=0x7fa6ec152e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:38.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.560+0000 7fa6f0f50640 1 -- 172.21.15.131:0/1158713502 shutdown_connections 2023-09-12T14:55:38.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.560+0000 7fa6f0f50640 1 -- 172.21.15.131:0/1158713502 wait complete. 2023-09-12T14:55:38.580 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-12T14:55:38.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.700+0000 7f1682f26640 1 -- 172.21.15.131:0/2824354025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f167c0cc430 msgr2=0x7f167c0c9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:38.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.700+0000 7f1682f26640 1 --2- 172.21.15.131:0/2824354025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f167c0cc430 0x7f167c0c9850 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f167c071110 tx=0x7f167401ccd0 comp rx=0 tx=0).stop 2023-09-12T14:55:38.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.700+0000 7f1682f26640 1 -- 172.21.15.131:0/2824354025 shutdown_connections 2023-09-12T14:55:38.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.700+0000 7f1682f26640 1 --2- 172.21.15.131:0/2824354025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f167c0cc430 0x7f167c0c9850 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:38.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.700+0000 7f1682f26640 1 --2- 172.21.15.131:0/2824354025 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f167c0abae0 0x7f167c0c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:38.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.700+0000 7f1682f26640 1 --2- 172.21.15.131:0/2824354025 >> v2:172.21.15.187:3300/0 conn(0x7f167c158d70 0x7f167c06b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:38.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.700+0000 7f1682f26640 1 -- 172.21.15.131:0/2824354025 >> 172.21.15.131:0/2824354025 conn(0x7f167c069840 msgr2=0x7f167c0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:38.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.700+0000 7f1682f26640 1 -- 172.21.15.131:0/2824354025 shutdown_connections 2023-09-12T14:55:38.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.700+0000 7f1682f26640 1 -- 172.21.15.131:0/2824354025 wait complete. 2023-09-12T14:55:38.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.700+0000 7f1682f26640 1 Processor -- start 2023-09-12T14:55:38.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.700+0000 7f1682f26640 1 -- start start 2023-09-12T14:55:38.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f1682f26640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f167c0abae0 0x7f167c153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:38.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f1682f26640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f167c0cc430 0x7f167c0ab0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:38.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f1682f26640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f167c158d70 0x7f167c1554a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:38.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f1682f26640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167c1572f0 con 0x7f167c0cc430 2023-09-12T14:55:38.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f1682f26640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167c0ab610 con 0x7f167c0abae0 2023-09-12T14:55:38.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f1682f26640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167c1566e0 con 0x7f167c158d70 2023-09-12T14:55:38.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f167bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f167c0cc430 0x7f167c0ab0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:38.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f167bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f167c0cc430 0x7f167c0ab0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48606/0 (socket says 172.21.15.131:48606) 2023-09-12T14:55:38.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f167bfff640 1 -- 172.21.15.131:0/3739424843 learned_addr learned my addr 172.21.15.131:0/3739424843 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:38.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f1680cc2640 1 --2- 172.21.15.131:0/3739424843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f167c0abae0 0x7f167c153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:38.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f16814c3640 1 --2- 172.21.15.131:0/3739424843 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f167c158d70 0x7f167c1554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:38.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f167bfff640 1 -- 172.21.15.131:0/3739424843 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f167c158d70 msgr2=0x7f167c1554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:38.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f167bfff640 1 --2- 172.21.15.131:0/3739424843 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f167c158d70 0x7f167c1554a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:38.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f167bfff640 1 -- 172.21.15.131:0/3739424843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f167c0abae0 msgr2=0x7f167c153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:38.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f167bfff640 1 --2- 172.21.15.131:0/3739424843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f167c0abae0 0x7f167c153f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:38.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f167bfff640 1 -- 172.21.15.131:0/3739424843 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16740098e0 con 0x7f167c0cc430 2023-09-12T14:55:38.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f1680cc2640 1 --2- 172.21.15.131:0/3739424843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f167c0abae0 0x7f167c153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:55:38.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f167bfff640 1 --2- 172.21.15.131:0/3739424843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f167c0cc430 0x7f167c0ab0d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f167000c940 tx=0x7f167000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:38.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f16797fa640 1 -- 172.21.15.131:0/3739424843 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670007b50 con 0x7f167c0cc430 2023-09-12T14:55:38.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f16797fa640 1 -- 172.21.15.131:0/3739424843 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1670007dc0 con 0x7f167c0cc430 2023-09-12T14:55:38.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f1682f26640 1 -- 172.21.15.131:0/3739424843 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f167c1569c0 con 0x7f167c0cc430 2023-09-12T14:55:38.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f16797fa640 1 -- 172.21.15.131:0/3739424843 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670005280 con 0x7f167c0cc430 2023-09-12T14:55:38.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f1682f26640 1 -- 172.21.15.131:0/3739424843 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f167c156f80 con 0x7f167c0cc430 2023-09-12T14:55:38.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f1682f26640 1 -- 172.21.15.131:0/3739424843 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f167c1f7a30 con 0x7f167c0cc430 2023-09-12T14:55:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f16797fa640 1 -- 172.21.15.131:0/3739424843 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f16700053e0 con 0x7f167c0cc430 2023-09-12T14:55:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f16797fa640 1 --2- 172.21.15.131:0/3739424843 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1668065b50 0x7f1668068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f16797fa640 1 -- 172.21.15.131:0/3739424843 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f1670083830 con 0x7f167c0cc430 2023-09-12T14:55:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f1680cc2640 1 --2- 172.21.15.131:0/3739424843 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1668065b50 0x7f1668068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.704+0000 7f1680cc2640 1 --2- 172.21.15.131:0/3739424843 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1668065b50 0x7f1668068010 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f166c005fd0 tx=0x7f166c00a540 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:38.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.708+0000 7f16797fa640 1 -- 172.21.15.131:0/3739424843 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f167c1f7a30 con 0x7f167c0cc430 2023-09-12T14:55:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.872+0000 7f1682f26640 1 -- 172.21.15.131:0/3739424843 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f167c1f7a30 con 0x7f167c0cc430 2023-09-12T14:55:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.872+0000 7f16797fa640 1 -- 172.21.15.131:0/3739424843 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f167c1f7a30 con 0x7f167c0cc430 2023-09-12T14:55:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2023-09-12T14:55:38.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.872+0000 7f1682f26640 1 -- 172.21.15.131:0/3739424843 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1668065b50 msgr2=0x7f1668068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:38.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.872+0000 7f1682f26640 1 --2- 172.21.15.131:0/3739424843 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1668065b50 0x7f1668068010 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f166c005fd0 tx=0x7f166c00a540 comp rx=0 tx=0).stop 2023-09-12T14:55:38.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.872+0000 7f1682f26640 1 -- 172.21.15.131:0/3739424843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f167c0cc430 msgr2=0x7f167c0ab0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:38.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.872+0000 7f1682f26640 1 --2- 172.21.15.131:0/3739424843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f167c0cc430 0x7f167c0ab0d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f167000c940 tx=0x7f167000ce00 comp rx=0 tx=0).stop 2023-09-12T14:55:38.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.872+0000 7f1682f26640 1 -- 172.21.15.131:0/3739424843 shutdown_connections 2023-09-12T14:55:38.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.872+0000 7f1682f26640 1 --2- 172.21.15.131:0/3739424843 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1668065b50 0x7f1668068010 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:38.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.872+0000 7f1682f26640 1 --2- 172.21.15.131:0/3739424843 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f167c158d70 0x7f167c1554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:38.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.872+0000 7f1682f26640 1 --2- 172.21.15.131:0/3739424843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f167c0cc430 0x7f167c0ab0d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:38.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.872+0000 7f1682f26640 1 --2- 172.21.15.131:0/3739424843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f167c0abae0 0x7f167c153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:38.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.872+0000 7f1682f26640 1 -- 172.21.15.131:0/3739424843 >> 172.21.15.131:0/3739424843 conn(0x7f167c069840 msgr2=0x7f167c14f190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:38.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.872+0000 7f1682f26640 1 -- 172.21.15.131:0/3739424843 shutdown_connections 2023-09-12T14:55:38.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:38.872+0000 7f1682f26640 1 -- 172.21.15.131:0/3739424843 wait complete. 2023-09-12T14:55:38.892 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:55:38.893 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-12T14:55:38.898 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:55:39.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd2d5e640 1 -- 172.21.15.131:0/318561730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ccc156930 msgr2=0x7f5ccc159dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:39.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd2d5e640 1 --2- 172.21.15.131:0/318561730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ccc156930 0x7f5ccc159dc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009840 tx=0x7f5cbc01c7d0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd2d5e640 1 -- 172.21.15.131:0/318561730 shutdown_connections 2023-09-12T14:55:39.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd2d5e640 1 --2- 172.21.15.131:0/318561730 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ccc0abb40 0x7f5ccc06b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd2d5e640 1 --2- 172.21.15.131:0/318561730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ccc156930 0x7f5ccc159dc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd2d5e640 1 -- 172.21.15.131:0/318561730 >> 172.21.15.131:0/318561730 conn(0x7f5ccc069840 msgr2=0x7f5ccc0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:39.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd2d5e640 1 -- 172.21.15.131:0/318561730 shutdown_connections 2023-09-12T14:55:39.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd2d5e640 1 -- 172.21.15.131:0/318561730 wait complete. 2023-09-12T14:55:39.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd2d5e640 1 Processor -- start 2023-09-12T14:55:39.071 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd2d5e640 1 -- start start 2023-09-12T14:55:39.071 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd2d5e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ccc0abb40 0x7f5ccc0ae450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:39.071 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd2d5e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ccc0c9370 0x7f5ccc14c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:39.071 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd2d5e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ccc156930 0x7f5ccc141a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:39.071 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd2d5e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ccc142930 con 0x7f5ccc156930 2023-09-12T14:55:39.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd2d5e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ccc0af4f0 con 0x7f5ccc0c9370 2023-09-12T14:55:39.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd2d5e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ccc150d90 con 0x7f5ccc0abb40 2023-09-12T14:55:39.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd12fb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ccc156930 0x7f5ccc141a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:39.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd12fb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ccc156930 0x7f5ccc141a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48612/0 (socket says 172.21.15.131:48612) 2023-09-12T14:55:39.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd12fb640 1 -- 172.21.15.131:0/654067927 learned_addr learned my addr 172.21.15.131:0/654067927 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5ccbfff640 1 --2- 172.21.15.131:0/654067927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ccc0c9370 0x7f5ccc14c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd0afa640 1 --2- 172.21.15.131:0/654067927 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ccc0abb40 0x7f5ccc0ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd12fb640 1 -- 172.21.15.131:0/654067927 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ccc0abb40 msgr2=0x7f5ccc0ae450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd12fb640 1 --2- 172.21.15.131:0/654067927 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ccc0abb40 0x7f5ccc0ae450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd12fb640 1 -- 172.21.15.131:0/654067927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ccc0c9370 msgr2=0x7f5ccc14c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:39.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd12fb640 1 --2- 172.21.15.131:0/654067927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ccc0c9370 0x7f5ccc14c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd12fb640 1 -- 172.21.15.131:0/654067927 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cbc0094a0 con 0x7f5ccc156930 2023-09-12T14:55:39.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd12fb640 1 --2- 172.21.15.131:0/654067927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ccc156930 0x7f5ccc141a50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400cb40 tx=0x7f5cc400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:39.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cc97fa640 1 -- 172.21.15.131:0/654067927 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc400aa50 con 0x7f5ccc156930 2023-09-12T14:55:39.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cc97fa640 1 -- 172.21.15.131:0/654067927 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5cc400aca0 con 0x7f5ccc156930 2023-09-12T14:55:39.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd2d5e640 1 -- 172.21.15.131:0/654067927 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ccc0ae990 con 0x7f5ccc156930 2023-09-12T14:55:39.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cc97fa640 1 -- 172.21.15.131:0/654067927 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4012680 con 0x7f5ccc156930 2023-09-12T14:55:39.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.064+0000 7f5cd2d5e640 1 -- 172.21.15.131:0/654067927 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ccc0aef30 con 0x7f5ccc156930 2023-09-12T14:55:39.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.068+0000 7f5cd2d5e640 1 -- 172.21.15.131:0/654067927 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ccc150f20 con 0x7f5ccc156930 2023-09-12T14:55:39.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.068+0000 7f5cc97fa640 1 -- 172.21.15.131:0/654067927 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f5cc401a3b0 con 0x7f5ccc156930 2023-09-12T14:55:39.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.068+0000 7f5cc97fa640 1 --2- 172.21.15.131:0/654067927 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5cb8065b20 0x7f5cb8067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:39.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.068+0000 7f5cd0afa640 1 --2- 172.21.15.131:0/654067927 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5cb8065b20 0x7f5cb8067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:39.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.068+0000 7f5cc97fa640 1 -- 172.21.15.131:0/654067927 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f5cc4083780 con 0x7f5ccc156930 2023-09-12T14:55:39.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.068+0000 7f5cd0afa640 1 --2- 172.21.15.131:0/654067927 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5cb8065b20 0x7f5cb8067fe0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc0af000 tx=0x7f5cbc01b010 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:39.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.072+0000 7f5cc97fa640 1 -- 172.21.15.131:0/654067927 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f5cc4051040 con 0x7f5ccc156930 2023-09-12T14:55:39.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.220+0000 7f5cd2d5e640 1 -- 172.21.15.131:0/654067927 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ccc073580 con 0x7f5ccc156930 2023-09-12T14:55:39.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.220+0000 7f5cc97fa640 1 -- 172.21.15.131:0/654067927 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16791 (secure 0 0 0) 0x7f5cc4054660 con 0x7f5ccc156930 2023-09-12T14:55:39.225 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:39.225 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":22,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:55:39.097847+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:54:55.689983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T14:54:57.029447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:55:39.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.224+0000 7f5cd2d5e640 1 -- 172.21.15.131:0/654067927 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5cb8065b20 msgr2=0x7f5cb8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:39.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.224+0000 7f5cd2d5e640 1 --2- 172.21.15.131:0/654067927 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5cb8065b20 0x7f5cb8067fe0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc0af000 tx=0x7f5cbc01b010 comp rx=0 tx=0).stop 2023-09-12T14:55:39.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.224+0000 7f5cd2d5e640 1 -- 172.21.15.131:0/654067927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ccc156930 msgr2=0x7f5ccc141a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:39.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.224+0000 7f5cd2d5e640 1 --2- 172.21.15.131:0/654067927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ccc156930 0x7f5ccc141a50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400cb40 tx=0x7f5cc400cc30 comp rx=0 tx=0).stop 2023-09-12T14:55:39.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.224+0000 7f5cd2d5e640 1 -- 172.21.15.131:0/654067927 shutdown_connections 2023-09-12T14:55:39.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.224+0000 7f5cd2d5e640 1 --2- 172.21.15.131:0/654067927 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5cb8065b20 0x7f5cb8067fe0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.224+0000 7f5cd2d5e640 1 --2- 172.21.15.131:0/654067927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ccc156930 0x7f5ccc141a50 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.224+0000 7f5cd2d5e640 1 --2- 172.21.15.131:0/654067927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ccc0c9370 0x7f5ccc14c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.224+0000 7f5cd2d5e640 1 --2- 172.21.15.131:0/654067927 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ccc0abb40 0x7f5ccc0ae450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.224+0000 7f5cd2d5e640 1 -- 172.21.15.131:0/654067927 >> 172.21.15.131:0/654067927 conn(0x7f5ccc069840 msgr2=0x7f5ccc142af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:39.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.224+0000 7f5cd2d5e640 1 -- 172.21.15.131:0/654067927 shutdown_connections 2023-09-12T14:55:39.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.224+0000 7f5cd2d5e640 1 -- 172.21.15.131:0/654067927 wait complete. 2023-09-12T14:55:39.243 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T14:54:45.654004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T14:54:55.689983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T14:54:57.029447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-12T14:55:39.244 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T14:55:39.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.356+0000 7f74322d6640 1 -- 172.21.15.131:0/2950787443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f742c15ad60 msgr2=0x7f742c0cb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:39.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.356+0000 7f74322d6640 1 --2- 172.21.15.131:0/2950787443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f742c15ad60 0x7f742c0cb6b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f742c0d0790 tx=0x7f741c01c830 comp rx=0 tx=0).stop 2023-09-12T14:55:39.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.356+0000 7f74322d6640 1 -- 172.21.15.131:0/2950787443 shutdown_connections 2023-09-12T14:55:39.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.356+0000 7f74322d6640 1 --2- 172.21.15.131:0/2950787443 >> v2:172.21.15.187:3300/0 conn(0x7f742c0b04a0 0x7f742c0cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.356+0000 7f74322d6640 1 --2- 172.21.15.131:0/2950787443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f742c15ad60 0x7f742c0cb6b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.356+0000 7f74322d6640 1 -- 172.21.15.131:0/2950787443 >> 172.21.15.131:0/2950787443 conn(0x7f742c069bb0 msgr2=0x7f742c069f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:39.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.360+0000 7f74322d6640 1 -- 172.21.15.131:0/2950787443 shutdown_connections 2023-09-12T14:55:39.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.360+0000 7f74322d6640 1 -- 172.21.15.131:0/2950787443 wait complete. 2023-09-12T14:55:39.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.360+0000 7f74322d6640 1 Processor -- start 2023-09-12T14:55:39.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.360+0000 7f74322d6640 1 -- start start 2023-09-12T14:55:39.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.360+0000 7f74322d6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f742c0b04a0 0x7f742c0b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:39.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.360+0000 7f74322d6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f742c0cbbf0 0x7f742c0c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:39.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.360+0000 7f74322d6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f742c15ad60 0x7f742c156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:39.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.360+0000 7f74322d6640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742c151540 con 0x7f742c0cbbf0 2023-09-12T14:55:39.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.360+0000 7f74322d6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742c151de0 con 0x7f742c0b04a0 2023-09-12T14:55:39.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.360+0000 7f74322d6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742c1544a0 con 0x7f742c15ad60 2023-09-12T14:55:39.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.360+0000 7f74237fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f742c0cbbf0 0x7f742c0c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:39.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.360+0000 7f74237fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f742c0cbbf0 0x7f742c0c8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48634/0 (socket says 172.21.15.131:48634) 2023-09-12T14:55:39.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.360+0000 7f74237fe640 1 -- 172.21.15.131:0/1137756985 learned_addr learned my addr 172.21.15.131:0/1137756985 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:39.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.360+0000 7f742bfff640 1 --2- 172.21.15.131:0/1137756985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f742c0b04a0 0x7f742c0b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:39.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.360+0000 7f7430873640 1 --2- 172.21.15.131:0/1137756985 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f742c15ad60 0x7f742c156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:39.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.360+0000 7f74237fe640 1 -- 172.21.15.131:0/1137756985 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f742c15ad60 msgr2=0x7f742c156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:39.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.360+0000 7f74237fe640 1 --2- 172.21.15.131:0/1137756985 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f742c15ad60 0x7f742c156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.360+0000 7f74237fe640 1 -- 172.21.15.131:0/1137756985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f742c0b04a0 msgr2=0x7f742c0b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:39.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.360+0000 7f74237fe640 1 --2- 172.21.15.131:0/1137756985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f742c0b04a0 0x7f742c0b08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.360+0000 7f74237fe640 1 -- 172.21.15.131:0/1137756985 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f741c009560 con 0x7f742c0cbbf0 2023-09-12T14:55:39.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.360+0000 7f74237fe640 1 --2- 172.21.15.131:0/1137756985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f742c0cbbf0 0x7f742c0c8b40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f741400c4c0 tx=0x7f7414001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:39.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.360+0000 7f74297fa640 1 -- 172.21.15.131:0/1137756985 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7414005480 con 0x7f742c0cbbf0 2023-09-12T14:55:39.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.360+0000 7f74297fa640 1 -- 172.21.15.131:0/1137756985 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7414003600 con 0x7f742c0cbbf0 2023-09-12T14:55:39.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.360+0000 7f74322d6640 1 -- 172.21.15.131:0/1137756985 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f742c156980 con 0x7f742c0cbbf0 2023-09-12T14:55:39.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.360+0000 7f74297fa640 1 -- 172.21.15.131:0/1137756985 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7414003a30 con 0x7f742c0cbbf0 2023-09-12T14:55:39.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.360+0000 7f74322d6640 1 -- 172.21.15.131:0/1137756985 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f742c151a70 con 0x7f742c0cbbf0 2023-09-12T14:55:39.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.360+0000 7f74322d6640 1 -- 172.21.15.131:0/1137756985 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f742c073580 con 0x7f742c0cbbf0 2023-09-12T14:55:39.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.364+0000 7f74297fa640 1 -- 172.21.15.131:0/1137756985 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f7414003b90 con 0x7f742c0cbbf0 2023-09-12T14:55:39.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.364+0000 7f74297fa640 1 --2- 172.21.15.131:0/1137756985 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f740c065b20 0x7f740c067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:39.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.364+0000 7f74297fa640 1 -- 172.21.15.131:0/1137756985 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f7414082770 con 0x7f742c0cbbf0 2023-09-12T14:55:39.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.364+0000 7f742bfff640 1 --2- 172.21.15.131:0/1137756985 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f740c065b20 0x7f740c067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:39.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.364+0000 7f742bfff640 1 --2- 172.21.15.131:0/1137756985 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f740c065b20 0x7f740c067fe0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f741c0af000 tx=0x7f741c01b070 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:39.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.368+0000 7f74297fa640 1 -- 172.21.15.131:0/1137756985 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f7414050050 con 0x7f742c0cbbf0 2023-09-12T14:55:39.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.508+0000 7f74322d6640 1 -- 172.21.15.131:0/1137756985 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f742c0cfb90 con 0x7f742c0cbbf0 2023-09-12T14:55:39.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.508+0000 7f74297fa640 1 -- 172.21.15.131:0/1137756985 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7414053670 con 0x7f742c0cbbf0 2023-09-12T14:55:39.511 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-09-12T14:55:39.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.508+0000 7f74322d6640 1 -- 172.21.15.131:0/1137756985 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f740c065b20 msgr2=0x7f740c067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:39.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.508+0000 7f74322d6640 1 --2- 172.21.15.131:0/1137756985 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f740c065b20 0x7f740c067fe0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f741c0af000 tx=0x7f741c01b070 comp rx=0 tx=0).stop 2023-09-12T14:55:39.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.508+0000 7f74322d6640 1 -- 172.21.15.131:0/1137756985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f742c0cbbf0 msgr2=0x7f742c0c8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:39.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.508+0000 7f74322d6640 1 --2- 172.21.15.131:0/1137756985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f742c0cbbf0 0x7f742c0c8b40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f741400c4c0 tx=0x7f7414001e80 comp rx=0 tx=0).stop 2023-09-12T14:55:39.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.508+0000 7f74322d6640 1 -- 172.21.15.131:0/1137756985 shutdown_connections 2023-09-12T14:55:39.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.508+0000 7f74322d6640 1 --2- 172.21.15.131:0/1137756985 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f740c065b20 0x7f740c067fe0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.508+0000 7f74322d6640 1 --2- 172.21.15.131:0/1137756985 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f742c15ad60 0x7f742c156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.508+0000 7f74322d6640 1 --2- 172.21.15.131:0/1137756985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f742c0cbbf0 0x7f742c0c8b40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.508+0000 7f74322d6640 1 --2- 172.21.15.131:0/1137756985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f742c0b04a0 0x7f742c0b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.508+0000 7f74322d6640 1 -- 172.21.15.131:0/1137756985 >> 172.21.15.131:0/1137756985 conn(0x7f742c069bb0 msgr2=0x7f742c15bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:39.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.508+0000 7f74322d6640 1 -- 172.21.15.131:0/1137756985 shutdown_connections 2023-09-12T14:55:39.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.508+0000 7f74322d6640 1 -- 172.21.15.131:0/1137756985 wait complete. 2023-09-12T14:55:39.528 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T14:55:39.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.652+0000 7f4058650640 1 -- 172.21.15.131:0/3132678246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f403c0047d0 msgr2=0x7f403c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:39.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.652+0000 7f4058650640 1 --2- 172.21.15.131:0/3132678246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f403c0047d0 0x7f403c004c50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f405006aa10 tx=0x7f404001c720 comp rx=0 tx=0).stop 2023-09-12T14:55:39.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.652+0000 7f4056bed640 1 -- 172.21.15.131:0/3132678246 reap_dead start 2023-09-12T14:55:39.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f4058650640 1 -- 172.21.15.131:0/3132678246 shutdown_connections 2023-09-12T14:55:39.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f4058650640 1 -- 172.21.15.131:0/3132678246 >> 172.21.15.131:0/3132678246 conn(0x7f40500c9b20 msgr2=0x7f40500c9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:39.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f4058650640 1 -- 172.21.15.131:0/3132678246 shutdown_connections 2023-09-12T14:55:39.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f4058650640 1 -- 172.21.15.131:0/3132678246 wait complete. 2023-09-12T14:55:39.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f4058650640 1 Processor -- start 2023-09-12T14:55:39.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f4058650640 1 -- start start 2023-09-12T14:55:39.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f4058650640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f403c0047d0 0x7f405014e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:39.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f4058650640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f405014c650 0x7f405014cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:39.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f4058650640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f405014a930 0x7f405014adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:39.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f4058650640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405014d510 con 0x7f403c0047d0 2023-09-12T14:55:39.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f4058650640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405014b2f0 con 0x7f405014a930 2023-09-12T14:55:39.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f4058650640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f405015ed90 con 0x7f405014c650 2023-09-12T14:55:39.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f40563ec640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f403c0047d0 0x7f405014e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:39.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f40563ec640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f403c0047d0 0x7f405014e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48646/0 (socket says 172.21.15.131:48646) 2023-09-12T14:55:39.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f40563ec640 1 -- 172.21.15.131:0/146290301 learned_addr learned my addr 172.21.15.131:0/146290301 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:39.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f4056bed640 1 --2- 172.21.15.131:0/146290301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f405014a930 0x7f405014adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:39.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f4055beb640 1 --2- 172.21.15.131:0/146290301 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f405014c650 0x7f405014cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:39.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f40563ec640 1 -- 172.21.15.131:0/146290301 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f405014c650 msgr2=0x7f405014cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:39.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f40563ec640 1 --2- 172.21.15.131:0/146290301 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f405014c650 0x7f405014cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f40563ec640 1 -- 172.21.15.131:0/146290301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f405014a930 msgr2=0x7f405014adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:39.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f40563ec640 1 --2- 172.21.15.131:0/146290301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f405014a930 0x7f405014adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f40563ec640 1 -- 172.21.15.131:0/146290301 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405015dcf0 con 0x7f403c0047d0 2023-09-12T14:55:39.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f4055beb640 1 --2- 172.21.15.131:0/146290301 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f405014c650 0x7f405014cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:39.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f40563ec640 1 --2- 172.21.15.131:0/146290301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f403c0047d0 0x7f405014e070 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f404c00e390 tx=0x7f404c00e850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:39.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f4046ffd640 1 -- 172.21.15.131:0/146290301 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c012910 con 0x7f403c0047d0 2023-09-12T14:55:39.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f4046ffd640 1 -- 172.21.15.131:0/146290301 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f404c012b60 con 0x7f403c0047d0 2023-09-12T14:55:39.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f4058650640 1 -- 172.21.15.131:0/146290301 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f405015dfd0 con 0x7f403c0047d0 2023-09-12T14:55:39.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f4046ffd640 1 -- 172.21.15.131:0/146290301 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c01a400 con 0x7f403c0047d0 2023-09-12T14:55:39.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f4058650640 1 -- 172.21.15.131:0/146290301 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f405015e590 con 0x7f403c0047d0 2023-09-12T14:55:39.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.656+0000 7f4058650640 1 -- 172.21.15.131:0/146290301 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f405006edd0 con 0x7f403c0047d0 2023-09-12T14:55:39.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.660+0000 7f4046ffd640 1 -- 172.21.15.131:0/146290301 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f404c01a560 con 0x7f403c0047d0 2023-09-12T14:55:39.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.660+0000 7f4046ffd640 1 --2- 172.21.15.131:0/146290301 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4034065b50 0x7f4034068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:39.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.660+0000 7f4046ffd640 1 -- 172.21.15.131:0/146290301 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f404c086a10 con 0x7f403c0047d0 2023-09-12T14:55:39.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.660+0000 7f4055beb640 1 --2- 172.21.15.131:0/146290301 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4034065b50 0x7f4034068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:39.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.660+0000 7f4055beb640 1 --2- 172.21.15.131:0/146290301 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4034065b50 0x7f4034068010 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f403c0057b0 tx=0x7f40400194c0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:39.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.664+0000 7f4046ffd640 1 -- 172.21.15.131:0/146290301 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f404c054370 con 0x7f403c0047d0 2023-09-12T14:55:39.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.804+0000 7f4058650640 1 -- 172.21.15.131:0/146290301 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f40500676c0 con 0x7f403c0047d0 2023-09-12T14:55:39.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.804+0000 7f4046ffd640 1 -- 172.21.15.131:0/146290301 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f404c01f0a0 con 0x7f403c0047d0 2023-09-12T14:55:39.808 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-09-12T14:55:39.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.808+0000 7f4058650640 1 -- 172.21.15.131:0/146290301 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4034065b50 msgr2=0x7f4034068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:39.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.808+0000 7f4058650640 1 --2- 172.21.15.131:0/146290301 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4034065b50 0x7f4034068010 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f403c0057b0 tx=0x7f40400194c0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.808+0000 7f4058650640 1 -- 172.21.15.131:0/146290301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f403c0047d0 msgr2=0x7f405014e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:39.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.808+0000 7f4058650640 1 --2- 172.21.15.131:0/146290301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f403c0047d0 0x7f405014e070 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f404c00e390 tx=0x7f404c00e850 comp rx=0 tx=0).stop 2023-09-12T14:55:39.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.808+0000 7f4058650640 1 -- 172.21.15.131:0/146290301 shutdown_connections 2023-09-12T14:55:39.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.808+0000 7f4058650640 1 --2- 172.21.15.131:0/146290301 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4034065b50 0x7f4034068010 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.808+0000 7f4058650640 1 --2- 172.21.15.131:0/146290301 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f405014c650 0x7f405014cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.808+0000 7f4058650640 1 --2- 172.21.15.131:0/146290301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f405014a930 0x7f405014adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.808+0000 7f4058650640 1 --2- 172.21.15.131:0/146290301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f403c0047d0 0x7f405014e070 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.808+0000 7f4058650640 1 -- 172.21.15.131:0/146290301 >> 172.21.15.131:0/146290301 conn(0x7f40500c9b20 msgr2=0x7f4050151040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:39.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.808+0000 7f4058650640 1 -- 172.21.15.131:0/146290301 shutdown_connections 2023-09-12T14:55:39.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.808+0000 7f4058650640 1 -- 172.21.15.131:0/146290301 wait complete. 2023-09-12T14:55:39.826 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T14:55:39.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8fc92640 1 -- 172.21.15.131:0/1950508389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb880ab730 msgr2=0x7fdb880abb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:39.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8fc92640 1 --2- 172.21.15.131:0/1950508389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb880ab730 0x7fdb880abb10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fdb8806cd90 tx=0x7fdb8401c870 comp rx=0 tx=0).stop 2023-09-12T14:55:39.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8fc92640 1 -- 172.21.15.131:0/1950508389 shutdown_connections 2023-09-12T14:55:39.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8fc92640 1 --2- 172.21.15.131:0/1950508389 >> v2:172.21.15.187:3300/0 conn(0x7fdb8806f390 0x7fdb8806f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8fc92640 1 --2- 172.21.15.131:0/1950508389 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdb88158180 0x7fdb880cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8fc92640 1 --2- 172.21.15.131:0/1950508389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb880ab730 0x7fdb880abb10 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8fc92640 1 -- 172.21.15.131:0/1950508389 >> 172.21.15.131:0/1950508389 conn(0x7fdb88069840 msgr2=0x7fdb880cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:39.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8fc92640 1 -- 172.21.15.131:0/1950508389 shutdown_connections 2023-09-12T14:55:39.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8fc92640 1 -- 172.21.15.131:0/1950508389 wait complete. 2023-09-12T14:55:39.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8fc92640 1 Processor -- start 2023-09-12T14:55:39.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8fc92640 1 -- start start 2023-09-12T14:55:39.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8fc92640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdb8806f390 0x7fdb8813a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:39.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8fc92640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb880ab730 0x7fdb88159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:39.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8fc92640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdb88158180 0x7fdb88159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:39.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8fc92640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb88136d60 con 0x7fdb880ab730 2023-09-12T14:55:39.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8fc92640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb88137f00 con 0x7fdb8806f390 2023-09-12T14:55:39.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8fc92640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb881372f0 con 0x7fdb88158180 2023-09-12T14:55:39.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8d22d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb880ab730 0x7fdb88159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:39.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8d22d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb880ab730 0x7fdb88159240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48660/0 (socket says 172.21.15.131:48660) 2023-09-12T14:55:39.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8d22d640 1 -- 172.21.15.131:0/2059125185 learned_addr learned my addr 172.21.15.131:0/2059125185 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:39.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8da2e640 1 --2- 172.21.15.131:0/2059125185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdb8806f390 0x7fdb8813a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:39.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8d22d640 1 -- 172.21.15.131:0/2059125185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdb88158180 msgr2=0x7fdb88159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:39.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8e22f640 1 --2- 172.21.15.131:0/2059125185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdb88158180 0x7fdb88159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:39.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8d22d640 1 --2- 172.21.15.131:0/2059125185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdb88158180 0x7fdb88159780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8d22d640 1 -- 172.21.15.131:0/2059125185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdb8806f390 msgr2=0x7fdb8813a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:39.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8d22d640 1 --2- 172.21.15.131:0/2059125185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdb8806f390 0x7fdb8813a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:39.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8d22d640 1 -- 172.21.15.131:0/2059125185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb84009560 con 0x7fdb880ab730 2023-09-12T14:55:39.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8da2e640 1 --2- 172.21.15.131:0/2059125185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdb8806f390 0x7fdb8813a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:39.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8d22d640 1 --2- 172.21.15.131:0/2059125185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb880ab730 0x7fdb88159240 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800c970 tx=0x7fdb7800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:39.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb7e7fc640 1 -- 172.21.15.131:0/2059125185 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb78004220 con 0x7fdb880ab730 2023-09-12T14:55:39.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb7e7fc640 1 -- 172.21.15.131:0/2059125185 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdb78004470 con 0x7fdb880ab730 2023-09-12T14:55:39.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb7e7fc640 1 -- 172.21.15.131:0/2059125185 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb78007c10 con 0x7fdb880ab730 2023-09-12T14:55:39.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8fc92640 1 -- 172.21.15.131:0/2059125185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb881375d0 con 0x7fdb880ab730 2023-09-12T14:55:39.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8fc92640 1 -- 172.21.15.131:0/2059125185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb88137b90 con 0x7fdb880ab730 2023-09-12T14:55:39.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.948+0000 7fdb8fc92640 1 -- 172.21.15.131:0/2059125185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb88073580 con 0x7fdb880ab730 2023-09-12T14:55:39.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.952+0000 7fdb7e7fc640 1 -- 172.21.15.131:0/2059125185 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fdb78003600 con 0x7fdb880ab730 2023-09-12T14:55:39.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.952+0000 7fdb7e7fc640 1 --2- 172.21.15.131:0/2059125185 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdb6c065b20 0x7fdb6c067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:39.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.952+0000 7fdb8da2e640 1 --2- 172.21.15.131:0/2059125185 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdb6c065b20 0x7fdb6c067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:39.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.952+0000 7fdb7e7fc640 1 -- 172.21.15.131:0/2059125185 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fdb780835d0 con 0x7fdb880ab730 2023-09-12T14:55:39.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.952+0000 7fdb8da2e640 1 --2- 172.21.15.131:0/2059125185 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdb6c065b20 0x7fdb6c067fe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdb840af000 tx=0x7fdb8401b0b0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:39.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:39.956+0000 7fdb7e7fc640 1 -- 172.21.15.131:0/2059125185 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fdb78050e20 con 0x7fdb880ab730 2023-09-12T14:55:40.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.108+0000 7fdb8fc92640 1 -- 172.21.15.131:0/2059125185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdb88152560 con 0x7fdb880ab730 2023-09-12T14:55:40.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.108+0000 7fdb7e7fc640 1 -- 172.21.15.131:0/2059125185 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdb78054440 con 0x7fdb880ab730 2023-09-12T14:55:40.111 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-09-12T14:55:40.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.108+0000 7fdb8fc92640 1 -- 172.21.15.131:0/2059125185 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdb6c065b20 msgr2=0x7fdb6c067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:40.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.108+0000 7fdb8fc92640 1 --2- 172.21.15.131:0/2059125185 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdb6c065b20 0x7fdb6c067fe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdb840af000 tx=0x7fdb8401b0b0 comp rx=0 tx=0).stop 2023-09-12T14:55:40.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.108+0000 7fdb8fc92640 1 -- 172.21.15.131:0/2059125185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb880ab730 msgr2=0x7fdb88159240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:40.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.108+0000 7fdb8fc92640 1 --2- 172.21.15.131:0/2059125185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb880ab730 0x7fdb88159240 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800c970 tx=0x7fdb7800ce30 comp rx=0 tx=0).stop 2023-09-12T14:55:40.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.108+0000 7fdb8fc92640 1 -- 172.21.15.131:0/2059125185 shutdown_connections 2023-09-12T14:55:40.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.108+0000 7fdb8fc92640 1 --2- 172.21.15.131:0/2059125185 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdb6c065b20 0x7fdb6c067fe0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:40.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.108+0000 7fdb8fc92640 1 --2- 172.21.15.131:0/2059125185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdb88158180 0x7fdb88159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:40.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.108+0000 7fdb8fc92640 1 --2- 172.21.15.131:0/2059125185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb880ab730 0x7fdb88159240 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:40.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.108+0000 7fdb8fc92640 1 --2- 172.21.15.131:0/2059125185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdb8806f390 0x7fdb8813a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:40.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.108+0000 7fdb8fc92640 1 -- 172.21.15.131:0/2059125185 >> 172.21.15.131:0/2059125185 conn(0x7fdb88069840 msgr2=0x7fdb880ade90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:40.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.108+0000 7fdb8fc92640 1 -- 172.21.15.131:0/2059125185 shutdown_connections 2023-09-12T14:55:40.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.108+0000 7fdb8fc92640 1 -- 172.21.15.131:0/2059125185 wait complete. 2023-09-12T14:55:40.130 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-12T14:55:40.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.252+0000 7f50d6462640 1 -- 172.21.15.131:0/2376871910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d015d570 msgr2=0x7f50d00cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:40.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.252+0000 7f50d6462640 1 --2- 172.21.15.131:0/2376871910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d015d570 0x7f50d00cb5d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f50d00ceb40 tx=0x7f50c001c700 comp rx=0 tx=0).stop 2023-09-12T14:55:40.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.252+0000 7f50d6462640 1 -- 172.21.15.131:0/2376871910 shutdown_connections 2023-09-12T14:55:40.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.252+0000 7f50d6462640 1 --2- 172.21.15.131:0/2376871910 >> v2:172.21.15.187:3300/0 conn(0x7f50d00cf700 0x7f50d00cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:40.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.252+0000 7f50d6462640 1 --2- 172.21.15.131:0/2376871910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d015d570 0x7f50d00cb5d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:40.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.252+0000 7f50d6462640 1 -- 172.21.15.131:0/2376871910 >> 172.21.15.131:0/2376871910 conn(0x7f50d00cebe0 msgr2=0x7f50d00ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:40.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.252+0000 7f50d6462640 1 -- 172.21.15.131:0/2376871910 shutdown_connections 2023-09-12T14:55:40.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.252+0000 7f50d6462640 1 -- 172.21.15.131:0/2376871910 wait complete. 2023-09-12T14:55:40.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.252+0000 7f50d6462640 1 Processor -- start 2023-09-12T14:55:40.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.252+0000 7f50d6462640 1 -- start start 2023-09-12T14:55:40.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50d6462640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d00b1ff0 0x7f50d006ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:40.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50d6462640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d00cf700 0x7f50d01508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:40.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50d6462640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d015d570 0x7f50d00c1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:40.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50d6462640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d006bc30 con 0x7f50d00cf700 2023-09-12T14:55:40.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50d6462640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d014ee80 con 0x7f50d015d570 2023-09-12T14:55:40.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50d6462640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d0150e20 con 0x7f50d00b1ff0 2023-09-12T14:55:40.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50cf7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d00cf700 0x7f50d01508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:40.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50cf7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d00cf700 0x7f50d01508e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48674/0 (socket says 172.21.15.131:48674) 2023-09-12T14:55:40.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50cf7fe640 1 -- 172.21.15.131:0/1848260526 learned_addr learned my addr 172.21.15.131:0/1848260526 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:40.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50cffff640 1 --2- 172.21.15.131:0/1848260526 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d00b1ff0 0x7f50d006ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:40.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50d49ff640 1 --2- 172.21.15.131:0/1848260526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d015d570 0x7f50d00c1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:40.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50cf7fe640 1 -- 172.21.15.131:0/1848260526 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d00b1ff0 msgr2=0x7f50d006ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:40.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50cf7fe640 1 --2- 172.21.15.131:0/1848260526 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d00b1ff0 0x7f50d006ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:40.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50cf7fe640 1 -- 172.21.15.131:0/1848260526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d015d570 msgr2=0x7f50d00c1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:40.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50cf7fe640 1 --2- 172.21.15.131:0/1848260526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d015d570 0x7f50d00c1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:40.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50cf7fe640 1 -- 172.21.15.131:0/1848260526 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c0009450 con 0x7f50d00cf700 2023-09-12T14:55:40.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50cffff640 1 --2- 172.21.15.131:0/1848260526 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d00b1ff0 0x7f50d006ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:40.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50cf7fe640 1 --2- 172.21.15.131:0/1848260526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d00cf700 0x7f50d01508e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f50c400d980 tx=0x7f50c400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:40.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50ccff9640 1 -- 172.21.15.131:0/1848260526 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c4009c30 con 0x7f50d00cf700 2023-09-12T14:55:40.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50ccff9640 1 -- 172.21.15.131:0/1848260526 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f50c4009e80 con 0x7f50d00cf700 2023-09-12T14:55:40.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50ccff9640 1 -- 172.21.15.131:0/1848260526 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c40106a0 con 0x7f50d00cf700 2023-09-12T14:55:40.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50d6462640 1 -- 172.21.15.131:0/1848260526 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d006be80 con 0x7f50d00cf700 2023-09-12T14:55:40.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50d6462640 1 -- 172.21.15.131:0/1848260526 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d00c1c60 con 0x7f50d00cf700 2023-09-12T14:55:40.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50d6462640 1 -- 172.21.15.131:0/1848260526 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d014f010 con 0x7f50d00cf700 2023-09-12T14:55:40.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50ccff9640 1 -- 172.21.15.131:0/1848260526 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f50c4010800 con 0x7f50d00cf700 2023-09-12T14:55:40.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50ccff9640 1 --2- 172.21.15.131:0/1848260526 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f50bc065b50 0x7f50bc068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:40.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50cffff640 1 --2- 172.21.15.131:0/1848260526 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f50bc065b50 0x7f50bc068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:40.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50ccff9640 1 -- 172.21.15.131:0/1848260526 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f50c4083860 con 0x7f50d00cf700 2023-09-12T14:55:40.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.256+0000 7f50cffff640 1 --2- 172.21.15.131:0/1848260526 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f50bc065b50 0x7f50bc068010 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f50c00ae000 tx=0x7f50c001af40 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:40.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.260+0000 7f50ccff9640 1 -- 172.21.15.131:0/1848260526 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f50c4051090 con 0x7f50d00cf700 2023-09-12T14:55:40.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.416+0000 7f50d6462640 1 -- 172.21.15.131:0/1848260526 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f50d0073580 con 0x7f50d00cf700 2023-09-12T14:55:40.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.416+0000 7f50ccff9640 1 -- 172.21.15.131:0/1848260526 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f50c4020080 con 0x7f50d00cf700 2023-09-12T14:55:40.421 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2023-09-12T14:55:40.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.420+0000 7f50d6462640 1 -- 172.21.15.131:0/1848260526 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f50bc065b50 msgr2=0x7f50bc068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:40.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.420+0000 7f50d6462640 1 --2- 172.21.15.131:0/1848260526 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f50bc065b50 0x7f50bc068010 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f50c00ae000 tx=0x7f50c001af40 comp rx=0 tx=0).stop 2023-09-12T14:55:40.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.420+0000 7f50d6462640 1 -- 172.21.15.131:0/1848260526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d00cf700 msgr2=0x7f50d01508e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:40.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.420+0000 7f50d6462640 1 --2- 172.21.15.131:0/1848260526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d00cf700 0x7f50d01508e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f50c400d980 tx=0x7f50c400de40 comp rx=0 tx=0).stop 2023-09-12T14:55:40.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.420+0000 7f50d6462640 1 -- 172.21.15.131:0/1848260526 shutdown_connections 2023-09-12T14:55:40.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.420+0000 7f50d6462640 1 --2- 172.21.15.131:0/1848260526 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f50bc065b50 0x7f50bc068010 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:40.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.420+0000 7f50d6462640 1 --2- 172.21.15.131:0/1848260526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d015d570 0x7f50d00c1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:40.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.420+0000 7f50d6462640 1 --2- 172.21.15.131:0/1848260526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d00cf700 0x7f50d01508e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:40.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.420+0000 7f50d6462640 1 --2- 172.21.15.131:0/1848260526 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d00b1ff0 0x7f50d006ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:40.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.420+0000 7f50d6462640 1 -- 172.21.15.131:0/1848260526 >> 172.21.15.131:0/1848260526 conn(0x7f50d00cebe0 msgr2=0x7f50d00cadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:40.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.420+0000 7f50d6462640 1 -- 172.21.15.131:0/1848260526 shutdown_connections 2023-09-12T14:55:40.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.420+0000 7f50d6462640 1 -- 172.21.15.131:0/1848260526 wait complete. 2023-09-12T14:55:40.440 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi131.front.sepia.ceph.com... 2023-09-12T14:55:40.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:40.441 DEBUG:teuthology.orchestra.run.smithi131:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-09-12T14:55:40.452 INFO:teuthology.orchestra.run:waiting for 300 2023-09-12T14:55:40.452 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2023-09-12T14:55:40.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:40.453 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-09-12T14:55:40.498 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:55:40.499 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-12T14:55:40.505 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:55:40.506 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-12T14:55:40.551 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:55:40.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.672+0000 7f617f19e640 1 -- 172.21.15.131:0/3580314041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6160003980 msgr2=0x7f6160003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:40.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.672+0000 7f617f19e640 1 --2- 172.21.15.131:0/3580314041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6160003980 0x7f6160003da0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f617000bb10 tx=0x7f617001cdc0 comp rx=0 tx=0).stop 2023-09-12T14:55:40.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.672+0000 7f617d73b640 1 -- 172.21.15.131:0/3580314041 reap_dead start 2023-09-12T14:55:40.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.672+0000 7f617f19e640 1 -- 172.21.15.131:0/3580314041 shutdown_connections 2023-09-12T14:55:40.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.672+0000 7f617f19e640 1 -- 172.21.15.131:0/3580314041 >> 172.21.15.131:0/3580314041 conn(0x7f6178069840 msgr2=0x7f61780cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:40.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.672+0000 7f617f19e640 1 -- 172.21.15.131:0/3580314041 shutdown_connections 2023-09-12T14:55:40.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.672+0000 7f617f19e640 1 -- 172.21.15.131:0/3580314041 wait complete. 2023-09-12T14:55:40.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f617f19e640 1 Processor -- start 2023-09-12T14:55:40.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f617f19e640 1 -- start start 2023-09-12T14:55:40.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f617f19e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6160003980 0x7f61780b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:40.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f617f19e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6178144700 0x7f61780b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:40.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f617f19e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6178157410 0x7f6178157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:40.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f617f19e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617806f5d0 con 0x7f6160003980 2023-09-12T14:55:40.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f617f19e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6178157dd0 con 0x7f6178157410 2023-09-12T14:55:40.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f617f19e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61780aad10 con 0x7f6178144700 2023-09-12T14:55:40.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f617cf3a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6160003980 0x7f61780b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:40.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f617cf3a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6160003980 0x7f61780b11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48706/0 (socket says 172.21.15.131:48706) 2023-09-12T14:55:40.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f617cf3a640 1 -- 172.21.15.131:0/949064373 learned_addr learned my addr 172.21.15.131:0/949064373 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:40.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f617d73b640 1 --2- 172.21.15.131:0/949064373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6178157410 0x7f6178157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:40.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f6177fff640 1 --2- 172.21.15.131:0/949064373 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6178144700 0x7f61780b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:40.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f617cf3a640 1 -- 172.21.15.131:0/949064373 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6178144700 msgr2=0x7f61780b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:40.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f617cf3a640 1 --2- 172.21.15.131:0/949064373 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6178144700 0x7f61780b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:40.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f617cf3a640 1 -- 172.21.15.131:0/949064373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6178157410 msgr2=0x7f6178157890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:40.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f617cf3a640 1 --2- 172.21.15.131:0/949064373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6178157410 0x7f6178157890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:40.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f617cf3a640 1 -- 172.21.15.131:0/949064373 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6178158020 con 0x7f6160003980 2023-09-12T14:55:40.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f617d73b640 1 --2- 172.21.15.131:0/949064373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6178157410 0x7f6178157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:55:40.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f617cf3a640 1 --2- 172.21.15.131:0/949064373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6160003980 0x7f61780b11e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f61700b1040 tx=0x7f61700a5f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:40.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f61757fa640 1 -- 172.21.15.131:0/949064373 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6170005300 con 0x7f6160003980 2023-09-12T14:55:40.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f61757fa640 1 -- 172.21.15.131:0/949064373 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6170005460 con 0x7f6160003980 2023-09-12T14:55:40.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f617f19e640 1 -- 172.21.15.131:0/949064373 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f617806eca0 con 0x7f6160003980 2023-09-12T14:55:40.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f61757fa640 1 -- 172.21.15.131:0/949064373 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6170004930 con 0x7f6160003980 2023-09-12T14:55:40.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f617f19e640 1 -- 172.21.15.131:0/949064373 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f617806f260 con 0x7f6160003980 2023-09-12T14:55:40.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f617f19e640 1 -- 172.21.15.131:0/949064373 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6178073580 con 0x7f6160003980 2023-09-12T14:55:40.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f61757fa640 1 -- 172.21.15.131:0/949064373 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f6170004a90 con 0x7f6160003980 2023-09-12T14:55:40.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f61757fa640 1 --2- 172.21.15.131:0/949064373 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6164065b50 0x7f6164068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:40.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f61757fa640 1 -- 172.21.15.131:0/949064373 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f617011dfc0 con 0x7f6160003980 2023-09-12T14:55:40.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.676+0000 7f6177fff640 1 --2- 172.21.15.131:0/949064373 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6164065b50 0x7f6164068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:40.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.680+0000 7f6177fff640 1 --2- 172.21.15.131:0/949064373 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6164065b50 0x7f6164068010 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f61780b1d80 tx=0x7f616c000ee0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:40.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.680+0000 7f61757fa640 1 -- 172.21.15.131:0/949064373 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f61700eb800 con 0x7f6160003980 2023-09-12T14:55:40.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.836+0000 7f617f19e640 1 -- 172.21.15.131:0/949064373 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61780cd4d0 con 0x7f6160003980 2023-09-12T14:55:40.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.840+0000 7f61757fa640 1 -- 172.21.15.131:0/949064373 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16791 (secure 0 0 0) 0x7f61700ab070 con 0x7f6160003980 2023-09-12T14:55:40.842 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:40.842 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":22,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:55:39.097847+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:54:55.689983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T14:54:57.029447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:55:40.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.840+0000 7f617f19e640 1 -- 172.21.15.131:0/949064373 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6164065b50 msgr2=0x7f6164068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:40.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.840+0000 7f617f19e640 1 --2- 172.21.15.131:0/949064373 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6164065b50 0x7f6164068010 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f61780b1d80 tx=0x7f616c000ee0 comp rx=0 tx=0).stop 2023-09-12T14:55:40.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.840+0000 7f617f19e640 1 -- 172.21.15.131:0/949064373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6160003980 msgr2=0x7f61780b11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:40.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.840+0000 7f617f19e640 1 --2- 172.21.15.131:0/949064373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6160003980 0x7f61780b11e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f61700b1040 tx=0x7f61700a5f10 comp rx=0 tx=0).stop 2023-09-12T14:55:40.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.840+0000 7f617f19e640 1 -- 172.21.15.131:0/949064373 shutdown_connections 2023-09-12T14:55:40.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.840+0000 7f617f19e640 1 --2- 172.21.15.131:0/949064373 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6164065b50 0x7f6164068010 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:40.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.840+0000 7f617f19e640 1 --2- 172.21.15.131:0/949064373 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6178144700 0x7f61780b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:40.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.840+0000 7f617f19e640 1 --2- 172.21.15.131:0/949064373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6178157410 0x7f6178157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:40.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.840+0000 7f617f19e640 1 --2- 172.21.15.131:0/949064373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6160003980 0x7f61780b11e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:40.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.840+0000 7f617f19e640 1 -- 172.21.15.131:0/949064373 >> 172.21.15.131:0/949064373 conn(0x7f6178069840 msgr2=0x7f6178144ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:40.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.840+0000 7f617f19e640 1 -- 172.21.15.131:0/949064373 shutdown_connections 2023-09-12T14:55:40.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.840+0000 7f617f19e640 1 -- 172.21.15.131:0/949064373 wait complete. 2023-09-12T14:55:40.861 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T14:54:45.654004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T14:54:55.689983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T14:54:57.029447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-12T14:55:40.861 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T14:55:40.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2d6544640 1 -- 172.21.15.131:0/1637044933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2bc005190 msgr2=0x7fe2bc00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:40.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2d6544640 1 --2- 172.21.15.131:0/1637044933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2bc005190 0x7fe2bc00b990 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe2d00ac0e0 tx=0x7fe2c801c740 comp rx=0 tx=0).stop 2023-09-12T14:55:40.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2d6544640 1 -- 172.21.15.131:0/1637044933 shutdown_connections 2023-09-12T14:55:40.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2d6544640 1 --2- 172.21.15.131:0/1637044933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2bc005190 0x7fe2bc00b990 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:40.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2d6544640 1 -- 172.21.15.131:0/1637044933 >> 172.21.15.131:0/1637044933 conn(0x7fe2d0069840 msgr2=0x7fe2d00cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:40.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2d6544640 1 -- 172.21.15.131:0/1637044933 shutdown_connections 2023-09-12T14:55:40.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2d6544640 1 -- 172.21.15.131:0/1637044933 wait complete. 2023-09-12T14:55:40.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2d6544640 1 Processor -- start 2023-09-12T14:55:40.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2d6544640 1 -- start start 2023-09-12T14:55:40.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2d6544640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2bc005190 0x7fe2d01414b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:40.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2d6544640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2d006eca0 0x7fe2d0153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:40.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2d6544640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2d006f120 0x7fe2d01541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:40.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2d6544640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d0070700 con 0x7fe2d006eca0 2023-09-12T14:55:40.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2d6544640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d006f660 con 0x7fe2bc005190 2023-09-12T14:55:40.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2d6544640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d006f940 con 0x7fe2d006f120 2023-09-12T14:55:40.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2cf7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2d006eca0 0x7fe2d0153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:40.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2cf7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2d006eca0 0x7fe2d0153c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48728/0 (socket says 172.21.15.131:48728) 2023-09-12T14:55:40.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2cf7fe640 1 -- 172.21.15.131:0/3775221126 learned_addr learned my addr 172.21.15.131:0/3775221126 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:40.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2cffff640 1 --2- 172.21.15.131:0/3775221126 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2bc005190 0x7fe2d01414b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:40.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2cf7fe640 1 -- 172.21.15.131:0/3775221126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2d006f120 msgr2=0x7fe2d01541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:40.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2cf7fe640 1 --2- 172.21.15.131:0/3775221126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2d006f120 0x7fe2d01541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:40.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2cf7fe640 1 -- 172.21.15.131:0/3775221126 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2bc005190 msgr2=0x7fe2d01414b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:40.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2cf7fe640 1 --2- 172.21.15.131:0/3775221126 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2bc005190 0x7fe2d01414b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:40.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2cf7fe640 1 -- 172.21.15.131:0/3775221126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d006fc20 con 0x7fe2d006eca0 2023-09-12T14:55:40.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2cf7fe640 1 --2- 172.21.15.131:0/3775221126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2d006eca0 0x7fe2d0153c60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8001000 tx=0x7fe2b800f820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:40.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2ccff9640 1 -- 172.21.15.131:0/3775221126 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b80084e0 con 0x7fe2d006eca0 2023-09-12T14:55:40.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2ccff9640 1 -- 172.21.15.131:0/3775221126 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe2b8006720 con 0x7fe2d006eca0 2023-09-12T14:55:40.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.980+0000 7fe2d6544640 1 -- 172.21.15.131:0/3775221126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d006ff00 con 0x7fe2d006eca0 2023-09-12T14:55:40.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.984+0000 7fe2ccff9640 1 -- 172.21.15.131:0/3775221126 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8006b30 con 0x7fe2d006eca0 2023-09-12T14:55:40.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.984+0000 7fe2d6544640 1 -- 172.21.15.131:0/3775221126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d01419f0 con 0x7fe2d006eca0 2023-09-12T14:55:40.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.984+0000 7fe2d6544640 1 -- 172.21.15.131:0/3775221126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d0073580 con 0x7fe2d006eca0 2023-09-12T14:55:40.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.984+0000 7fe2ccff9640 1 -- 172.21.15.131:0/3775221126 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fe2b8006c90 con 0x7fe2d006eca0 2023-09-12T14:55:40.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.984+0000 7fe2ccff9640 1 --2- 172.21.15.131:0/3775221126 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe2b0065b20 0x7fe2b0067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:40.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.984+0000 7fe2ccff9640 1 -- 172.21.15.131:0/3775221126 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe2b8085a40 con 0x7fe2d006eca0 2023-09-12T14:55:40.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.984+0000 7fe2cffff640 1 --2- 172.21.15.131:0/3775221126 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe2b0065b20 0x7fe2b0067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:40.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.984+0000 7fe2cffff640 1 --2- 172.21.15.131:0/3775221126 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe2b0065b20 0x7fe2b0067fe0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0005b50 tx=0x7fe2c0009940 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:40.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:40.988+0000 7fe2ccff9640 1 -- 172.21.15.131:0/3775221126 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fe2b8087e60 con 0x7fe2d006eca0 2023-09-12T14:55:41.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.140+0000 7fe2d6544640 1 -- 172.21.15.131:0/3775221126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe2d00c8ee0 con 0x7fe2d006eca0 2023-09-12T14:55:41.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.140+0000 7fe2ccff9640 1 -- 172.21.15.131:0/3775221126 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe2b801a020 con 0x7fe2d006eca0 2023-09-12T14:55:41.144 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-09-12T14:55:41.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.144+0000 7fe2d6544640 1 -- 172.21.15.131:0/3775221126 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe2b0065b20 msgr2=0x7fe2b0067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:41.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.144+0000 7fe2d6544640 1 --2- 172.21.15.131:0/3775221126 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe2b0065b20 0x7fe2b0067fe0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0005b50 tx=0x7fe2c0009940 comp rx=0 tx=0).stop 2023-09-12T14:55:41.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.144+0000 7fe2d6544640 1 -- 172.21.15.131:0/3775221126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2d006eca0 msgr2=0x7fe2d0153c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:41.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.144+0000 7fe2d6544640 1 --2- 172.21.15.131:0/3775221126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2d006eca0 0x7fe2d0153c60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8001000 tx=0x7fe2b800f820 comp rx=0 tx=0).stop 2023-09-12T14:55:41.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.144+0000 7fe2d6544640 1 -- 172.21.15.131:0/3775221126 shutdown_connections 2023-09-12T14:55:41.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.144+0000 7fe2d6544640 1 --2- 172.21.15.131:0/3775221126 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe2b0065b20 0x7fe2b0067fe0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:41.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.144+0000 7fe2d6544640 1 --2- 172.21.15.131:0/3775221126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2d006f120 0x7fe2d01541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:41.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.144+0000 7fe2d6544640 1 --2- 172.21.15.131:0/3775221126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2d006eca0 0x7fe2d0153c60 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:41.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.144+0000 7fe2d6544640 1 --2- 172.21.15.131:0/3775221126 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2bc005190 0x7fe2d01414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:41.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.144+0000 7fe2d6544640 1 -- 172.21.15.131:0/3775221126 >> 172.21.15.131:0/3775221126 conn(0x7fe2d0069840 msgr2=0x7fe2d00ad980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:41.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.144+0000 7fe2d6544640 1 -- 172.21.15.131:0/3775221126 shutdown_connections 2023-09-12T14:55:41.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.144+0000 7fe2d6544640 1 -- 172.21.15.131:0/3775221126 wait complete. 2023-09-12T14:55:41.162 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T14:55:41.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.288+0000 7f9a79d3d640 1 -- 172.21.15.131:0/4147986150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a740abb40 msgr2=0x7f9a7406b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:41.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.288+0000 7f9a79d3d640 1 --2- 172.21.15.131:0/4147986150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a740abb40 0x7f9a7406b4a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009840 tx=0x7f9a6801c7d0 comp rx=0 tx=0).stop 2023-09-12T14:55:41.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.288+0000 7f9a79d3d640 1 -- 172.21.15.131:0/4147986150 shutdown_connections 2023-09-12T14:55:41.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.288+0000 7f9a79d3d640 1 --2- 172.21.15.131:0/4147986150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a740abb40 0x7f9a7406b4a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:41.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.288+0000 7f9a79d3d640 1 -- 172.21.15.131:0/4147986150 >> 172.21.15.131:0/4147986150 conn(0x7f9a74069840 msgr2=0x7f9a740cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:41.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.288+0000 7f9a79d3d640 1 -- 172.21.15.131:0/4147986150 shutdown_connections 2023-09-12T14:55:41.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.288+0000 7f9a79d3d640 1 -- 172.21.15.131:0/4147986150 wait complete. 2023-09-12T14:55:41.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.288+0000 7f9a79d3d640 1 Processor -- start 2023-09-12T14:55:41.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.288+0000 7f9a79d3d640 1 -- start start 2023-09-12T14:55:41.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.288+0000 7f9a79d3d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a740abb40 0x7f9a740ad770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:41.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.288+0000 7f9a79d3d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a740c9370 0x7f9a74144a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:41.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.288+0000 7f9a79d3d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a74156930 0x7f9a741441b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:41.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.288+0000 7f9a79d3d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a740af7f0 con 0x7f9a74156930 2023-09-12T14:55:41.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.288+0000 7f9a79d3d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7415abe0 con 0x7f9a740abb40 2023-09-12T14:55:41.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.288+0000 7f9a79d3d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7415aec0 con 0x7f9a740c9370 2023-09-12T14:55:41.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.288+0000 7f9a73fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a74156930 0x7f9a741441b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:41.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.288+0000 7f9a73fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a74156930 0x7f9a741441b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48740/0 (socket says 172.21.15.131:48740) 2023-09-12T14:55:41.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.288+0000 7f9a73fff640 1 -- 172.21.15.131:0/2141482717 learned_addr learned my addr 172.21.15.131:0/2141482717 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:41.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.288+0000 7f9a72ffd640 1 --2- 172.21.15.131:0/2141482717 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a740c9370 0x7f9a74144a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:41.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.288+0000 7f9a737fe640 1 --2- 172.21.15.131:0/2141482717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a740abb40 0x7f9a740ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:41.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.288+0000 7f9a73fff640 1 -- 172.21.15.131:0/2141482717 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a740c9370 msgr2=0x7f9a74144a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:41.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.288+0000 7f9a73fff640 1 --2- 172.21.15.131:0/2141482717 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a740c9370 0x7f9a74144a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:41.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.288+0000 7f9a73fff640 1 -- 172.21.15.131:0/2141482717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a740abb40 msgr2=0x7f9a740ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:41.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.288+0000 7f9a73fff640 1 --2- 172.21.15.131:0/2141482717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a740abb40 0x7f9a740ad770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:41.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.292+0000 7f9a73fff640 1 -- 172.21.15.131:0/2141482717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a680094a0 con 0x7f9a74156930 2023-09-12T14:55:41.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.292+0000 7f9a73fff640 1 --2- 172.21.15.131:0/2141482717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a74156930 0x7f9a741441b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00cbc0 tx=0x7f9a6c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:41.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.292+0000 7f9a5bfff640 1 -- 172.21.15.131:0/2141482717 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c00cea0 con 0x7f9a74156930 2023-09-12T14:55:41.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.292+0000 7f9a5bfff640 1 -- 172.21.15.131:0/2141482717 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9a6c00ab40 con 0x7f9a74156930 2023-09-12T14:55:41.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.292+0000 7f9a79d3d640 1 -- 172.21.15.131:0/2141482717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a7415b1a0 con 0x7f9a74156930 2023-09-12T14:55:41.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.292+0000 7f9a5bfff640 1 -- 172.21.15.131:0/2141482717 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c012420 con 0x7f9a74156930 2023-09-12T14:55:41.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.292+0000 7f9a79d3d640 1 -- 172.21.15.131:0/2141482717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a74144fa0 con 0x7f9a74156930 2023-09-12T14:55:41.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.292+0000 7f9a79d3d640 1 -- 172.21.15.131:0/2141482717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a74073580 con 0x7f9a74156930 2023-09-12T14:55:41.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.292+0000 7f9a5bfff640 1 -- 172.21.15.131:0/2141482717 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f9a6c01a3b0 con 0x7f9a74156930 2023-09-12T14:55:41.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.292+0000 7f9a5bfff640 1 --2- 172.21.15.131:0/2141482717 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9a60065b20 0x7f9a60067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:41.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.292+0000 7f9a5bfff640 1 -- 172.21.15.131:0/2141482717 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f9a6c0837c0 con 0x7f9a74156930 2023-09-12T14:55:41.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.292+0000 7f9a737fe640 1 --2- 172.21.15.131:0/2141482717 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9a60065b20 0x7f9a60067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:41.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.292+0000 7f9a737fe640 1 --2- 172.21.15.131:0/2141482717 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9a60065b20 0x7f9a60067fe0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9a64007fd0 tx=0x7f9a64009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:41.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.296+0000 7f9a5bfff640 1 -- 172.21.15.131:0/2141482717 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f9a6c051090 con 0x7f9a74156930 2023-09-12T14:55:41.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.440+0000 7f9a79d3d640 1 -- 172.21.15.131:0/2141482717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9a740b1b30 con 0x7f9a74156930 2023-09-12T14:55:41.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.440+0000 7f9a5bfff640 1 -- 172.21.15.131:0/2141482717 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9a6c0546b0 con 0x7f9a74156930 2023-09-12T14:55:41.446 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-09-12T14:55:41.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.444+0000 7f9a79d3d640 1 -- 172.21.15.131:0/2141482717 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9a60065b20 msgr2=0x7f9a60067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:41.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.444+0000 7f9a79d3d640 1 --2- 172.21.15.131:0/2141482717 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9a60065b20 0x7f9a60067fe0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9a64007fd0 tx=0x7f9a64009040 comp rx=0 tx=0).stop 2023-09-12T14:55:41.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.444+0000 7f9a79d3d640 1 -- 172.21.15.131:0/2141482717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a74156930 msgr2=0x7f9a741441b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:41.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.444+0000 7f9a79d3d640 1 --2- 172.21.15.131:0/2141482717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a74156930 0x7f9a741441b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00cbc0 tx=0x7f9a6c00ccb0 comp rx=0 tx=0).stop 2023-09-12T14:55:41.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.444+0000 7f9a79d3d640 1 -- 172.21.15.131:0/2141482717 shutdown_connections 2023-09-12T14:55:41.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.444+0000 7f9a79d3d640 1 --2- 172.21.15.131:0/2141482717 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9a60065b20 0x7f9a60067fe0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:41.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.444+0000 7f9a79d3d640 1 --2- 172.21.15.131:0/2141482717 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a740c9370 0x7f9a74144a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:41.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.444+0000 7f9a79d3d640 1 --2- 172.21.15.131:0/2141482717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a74156930 0x7f9a741441b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:41.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.444+0000 7f9a79d3d640 1 --2- 172.21.15.131:0/2141482717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a740abb40 0x7f9a740ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:41.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.444+0000 7f9a79d3d640 1 -- 172.21.15.131:0/2141482717 >> 172.21.15.131:0/2141482717 conn(0x7f9a74069840 msgr2=0x7f9a7414e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:41.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.444+0000 7f9a79d3d640 1 -- 172.21.15.131:0/2141482717 shutdown_connections 2023-09-12T14:55:41.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.444+0000 7f9a79d3d640 1 -- 172.21.15.131:0/2141482717 wait complete. 2023-09-12T14:55:41.463 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T14:55:41.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7faffff640 1 -- 172.21.15.131:0/4140684397 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa80ad440 con 0x7f7fb80d1680 2023-09-12T14:55:41.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbf9ac640 1 -- 172.21.15.131:0/4140684397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fb80d1680 msgr2=0x7f7fb815e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:41.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbf9ac640 1 --2- 172.21.15.131:0/4140684397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fb80d1680 0x7f7fb815e820 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8009870 tx=0x7f7fa801c800 comp rx=0 tx=0).stop 2023-09-12T14:55:41.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbf9ac640 1 -- 172.21.15.131:0/4140684397 shutdown_connections 2023-09-12T14:55:41.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbf9ac640 1 --2- 172.21.15.131:0/4140684397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fb80d1680 0x7f7fb815e820 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:41.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbf9ac640 1 --2- 172.21.15.131:0/4140684397 >> v2:172.21.15.187:3300/0 conn(0x7f7fb80cf2d0 0x7f7fb815e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:41.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbf9ac640 1 -- 172.21.15.131:0/4140684397 >> 172.21.15.131:0/4140684397 conn(0x7f7fb809a200 msgr2=0x7f7fb80d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:41.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbf9ac640 1 -- 172.21.15.131:0/4140684397 shutdown_connections 2023-09-12T14:55:41.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbf9ac640 1 -- 172.21.15.131:0/4140684397 wait complete. 2023-09-12T14:55:41.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbf9ac640 1 Processor -- start 2023-09-12T14:55:41.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbf9ac640 1 -- start start 2023-09-12T14:55:41.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbf9ac640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fb80cf2d0 0x7f7fb80c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:41.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbf9ac640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fb80cf6b0 0x7f7fb80c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:41.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbf9ac640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fb80d1680 0x7f7fb815b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:41.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbf9ac640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb80b3f20 con 0x7f7fb80cf6b0 2023-09-12T14:55:41.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbf9ac640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb80c8580 con 0x7f7fb80cf2d0 2023-09-12T14:55:41.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbf9ac640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb80c7d00 con 0x7f7fb80d1680 2023-09-12T14:55:41.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbcf47640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fb80cf6b0 0x7f7fb80c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:41.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbcf47640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fb80cf6b0 0x7f7fb80c77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48756/0 (socket says 172.21.15.131:48756) 2023-09-12T14:55:41.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbcf47640 1 -- 172.21.15.131:0/3990645881 learned_addr learned my addr 172.21.15.131:0/3990645881 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:41.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbdf49640 1 --2- 172.21.15.131:0/3990645881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fb80d1680 0x7f7fb815b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:41.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbd748640 1 --2- 172.21.15.131:0/3990645881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fb80cf2d0 0x7f7fb80c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:41.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbcf47640 1 -- 172.21.15.131:0/3990645881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fb80d1680 msgr2=0x7f7fb815b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:41.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbcf47640 1 --2- 172.21.15.131:0/3990645881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fb80d1680 0x7f7fb815b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:41.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbcf47640 1 -- 172.21.15.131:0/3990645881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fb80cf2d0 msgr2=0x7f7fb80c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:41.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbcf47640 1 --2- 172.21.15.131:0/3990645881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fb80cf2d0 0x7f7fb80c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:41.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbcf47640 1 -- 172.21.15.131:0/3990645881 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb0009820 con 0x7f7fb80cf6b0 2023-09-12T14:55:41.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbdf49640 1 --2- 172.21.15.131:0/3990645881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fb80d1680 0x7f7fb815b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:41.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.576+0000 7f7fbcf47640 1 --2- 172.21.15.131:0/3990645881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fb80cf6b0 0x7f7fb80c77c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8000c10 tx=0x7f7fa8005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:41.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.580+0000 7f7fadffb640 1 -- 172.21.15.131:0/3990645881 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa80bc070 con 0x7f7fb80cf6b0 2023-09-12T14:55:41.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.580+0000 7f7fadffb640 1 -- 172.21.15.131:0/3990645881 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7fa80a5830 con 0x7f7fb80cf6b0 2023-09-12T14:55:41.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.580+0000 7f7fbf9ac640 1 -- 172.21.15.131:0/3990645881 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa80094a0 con 0x7f7fb80cf6b0 2023-09-12T14:55:41.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.580+0000 7f7fadffb640 1 -- 172.21.15.131:0/3990645881 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa80048d0 con 0x7f7fb80cf6b0 2023-09-12T14:55:41.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.580+0000 7f7fbf9ac640 1 -- 172.21.15.131:0/3990645881 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb80b3b60 con 0x7f7fb80cf6b0 2023-09-12T14:55:41.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.580+0000 7f7fbf9ac640 1 -- 172.21.15.131:0/3990645881 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb8073580 con 0x7f7fb80cf6b0 2023-09-12T14:55:41.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.580+0000 7f7fadffb640 1 -- 172.21.15.131:0/3990645881 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f7fa80b4040 con 0x7f7fb80cf6b0 2023-09-12T14:55:41.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.580+0000 7f7fadffb640 1 --2- 172.21.15.131:0/3990645881 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7f9c065b20 0x7f7f9c067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:41.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.580+0000 7f7fadffb640 1 -- 172.21.15.131:0/3990645881 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f7fa811dbb0 con 0x7f7fb80cf6b0 2023-09-12T14:55:41.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.580+0000 7f7fbd748640 1 --2- 172.21.15.131:0/3990645881 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7f9c065b20 0x7f7f9c067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:41.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.580+0000 7f7fbd748640 1 --2- 172.21.15.131:0/3990645881 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7f9c065b20 0x7f7f9c067fe0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7fb000f360 tx=0x7f7fb0009500 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:41.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.584+0000 7f7fadffb640 1 -- 172.21.15.131:0/3990645881 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f7fa80eb510 con 0x7f7fb80cf6b0 2023-09-12T14:55:41.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.724+0000 7f7fbf9ac640 1 -- 172.21.15.131:0/3990645881 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7fb80d0250 con 0x7f7fb80cf6b0 2023-09-12T14:55:41.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.728+0000 7f7fadffb640 1 -- 172.21.15.131:0/3990645881 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7fa80b20a0 con 0x7f7fb80cf6b0 2023-09-12T14:55:41.730 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-09-12T14:55:41.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.728+0000 7f7fbf9ac640 1 -- 172.21.15.131:0/3990645881 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7f9c065b20 msgr2=0x7f7f9c067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:41.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.728+0000 7f7fbf9ac640 1 --2- 172.21.15.131:0/3990645881 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7f9c065b20 0x7f7f9c067fe0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7fb000f360 tx=0x7f7fb0009500 comp rx=0 tx=0).stop 2023-09-12T14:55:41.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.728+0000 7f7fbf9ac640 1 -- 172.21.15.131:0/3990645881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fb80cf6b0 msgr2=0x7f7fb80c77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:41.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.728+0000 7f7fbf9ac640 1 --2- 172.21.15.131:0/3990645881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fb80cf6b0 0x7f7fb80c77c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8000c10 tx=0x7f7fa8005750 comp rx=0 tx=0).stop 2023-09-12T14:55:41.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.728+0000 7f7fbf9ac640 1 -- 172.21.15.131:0/3990645881 shutdown_connections 2023-09-12T14:55:41.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.728+0000 7f7fbf9ac640 1 --2- 172.21.15.131:0/3990645881 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7f9c065b20 0x7f7f9c067fe0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:41.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.728+0000 7f7fbf9ac640 1 --2- 172.21.15.131:0/3990645881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fb80d1680 0x7f7fb815b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:41.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.728+0000 7f7fbf9ac640 1 --2- 172.21.15.131:0/3990645881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fb80cf6b0 0x7f7fb80c77c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:41.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.728+0000 7f7fbf9ac640 1 --2- 172.21.15.131:0/3990645881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fb80cf2d0 0x7f7fb80c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:41.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.728+0000 7f7fbf9ac640 1 -- 172.21.15.131:0/3990645881 >> 172.21.15.131:0/3990645881 conn(0x7f7fb809a200 msgr2=0x7f7fb80cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:41.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.728+0000 7f7fbf9ac640 1 -- 172.21.15.131:0/3990645881 shutdown_connections 2023-09-12T14:55:41.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.728+0000 7f7fbf9ac640 1 -- 172.21.15.131:0/3990645881 wait complete. 2023-09-12T14:55:41.748 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-12T14:55:41.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.876+0000 7f0cda62f640 1 -- 172.21.15.131:0/3966795221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cd40d1680 msgr2=0x7f0cd40b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:41.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.876+0000 7f0cda62f640 1 --2- 172.21.15.131:0/3966795221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cd40d1680 0x7f0cd40b0750 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8009900 tx=0x7f0cc801c890 comp rx=0 tx=0).stop 2023-09-12T14:55:41.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.876+0000 7f0cda62f640 1 -- 172.21.15.131:0/3966795221 shutdown_connections 2023-09-12T14:55:41.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.876+0000 7f0cda62f640 1 --2- 172.21.15.131:0/3966795221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cd415e9f0 0x7f0cd40b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:41.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.876+0000 7f0cda62f640 1 --2- 172.21.15.131:0/3966795221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cd40d1680 0x7f0cd40b0750 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:41.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.876+0000 7f0cda62f640 1 -- 172.21.15.131:0/3966795221 >> 172.21.15.131:0/3966795221 conn(0x7f0cd409a200 msgr2=0x7f0cd40d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:41.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.876+0000 7f0cda62f640 1 -- 172.21.15.131:0/3966795221 shutdown_connections 2023-09-12T14:55:41.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.876+0000 7f0cda62f640 1 -- 172.21.15.131:0/3966795221 wait complete. 2023-09-12T14:55:41.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.876+0000 7f0cda62f640 1 Processor -- start 2023-09-12T14:55:41.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.876+0000 7f0cda62f640 1 -- start start 2023-09-12T14:55:41.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cda62f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cd40cf2d0 0x7f0cd4151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:41.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cda62f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cd40d1680 0x7f0cd41509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:41.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cda62f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cd415e9f0 0x7f0cd406ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:41.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cda62f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd4154c80 con 0x7f0cd415e9f0 2023-09-12T14:55:41.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cda62f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd4151870 con 0x7f0cd40d1680 2023-09-12T14:55:41.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cda62f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd4151b50 con 0x7f0cd40cf2d0 2023-09-12T14:55:41.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cd8bcc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cd415e9f0 0x7f0cd406ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:41.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cd8bcc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cd415e9f0 0x7f0cd406ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48774/0 (socket says 172.21.15.131:48774) 2023-09-12T14:55:41.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cd8bcc640 1 -- 172.21.15.131:0/1575747248 learned_addr learned my addr 172.21.15.131:0/1575747248 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:41.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cd37fe640 1 --2- 172.21.15.131:0/1575747248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cd40d1680 0x7f0cd41509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:41.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cd3fff640 1 --2- 172.21.15.131:0/1575747248 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cd40cf2d0 0x7f0cd4151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:41.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cd8bcc640 1 -- 172.21.15.131:0/1575747248 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cd40cf2d0 msgr2=0x7f0cd4151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:41.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cd8bcc640 1 --2- 172.21.15.131:0/1575747248 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cd40cf2d0 0x7f0cd4151170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:41.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cd8bcc640 1 -- 172.21.15.131:0/1575747248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cd40d1680 msgr2=0x7f0cd41509d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:41.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cd8bcc640 1 --2- 172.21.15.131:0/1575747248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cd40d1680 0x7f0cd41509d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:41.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cd8bcc640 1 -- 172.21.15.131:0/1575747248 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc8009560 con 0x7f0cd415e9f0 2023-09-12T14:55:41.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cd3fff640 1 --2- 172.21.15.131:0/1575747248 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cd40cf2d0 0x7f0cd4151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:41.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cd8bcc640 1 --2- 172.21.15.131:0/1575747248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cd415e9f0 0x7f0cd406ab50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc00e970 tx=0x7f0ccc00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:41.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cd0ff9640 1 -- 172.21.15.131:0/1575747248 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc004d30 con 0x7f0cd415e9f0 2023-09-12T14:55:41.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cd0ff9640 1 -- 172.21.15.131:0/1575747248 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0ccc00cb20 con 0x7f0cd415e9f0 2023-09-12T14:55:41.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cda62f640 1 -- 172.21.15.131:0/1575747248 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd4151e30 con 0x7f0cd415e9f0 2023-09-12T14:55:41.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cd0ff9640 1 -- 172.21.15.131:0/1575747248 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc005020 con 0x7f0cd415e9f0 2023-09-12T14:55:41.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cda62f640 1 -- 172.21.15.131:0/1575747248 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd41523f0 con 0x7f0cd415e9f0 2023-09-12T14:55:41.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cda62f640 1 -- 172.21.15.131:0/1575747248 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd4154e80 con 0x7f0cd415e9f0 2023-09-12T14:55:41.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cd0ff9640 1 -- 172.21.15.131:0/1575747248 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f0ccc005180 con 0x7f0cd415e9f0 2023-09-12T14:55:41.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cd0ff9640 1 --2- 172.21.15.131:0/1575747248 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0cc0065b20 0x7f0cc0067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:41.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cd0ff9640 1 -- 172.21.15.131:0/1575747248 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f0ccc083d00 con 0x7f0cd415e9f0 2023-09-12T14:55:41.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.880+0000 7f0cd3fff640 1 --2- 172.21.15.131:0/1575747248 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0cc0065b20 0x7f0cc0067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:41.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.884+0000 7f0cd3fff640 1 --2- 172.21.15.131:0/1575747248 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0cc0065b20 0x7f0cc0067fe0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40cc900 tx=0x7f0cc4009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:41.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:41.888+0000 7f0cd0ff9640 1 -- 172.21.15.131:0/1575747248 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f0cd4154e80 con 0x7f0cd415e9f0 2023-09-12T14:55:42.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.052+0000 7f0cda62f640 1 -- 172.21.15.131:0/1575747248 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0cd4154e80 con 0x7f0cd415e9f0 2023-09-12T14:55:42.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.052+0000 7f0cd0ff9640 1 -- 172.21.15.131:0/1575747248 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0cd4154e80 con 0x7f0cd415e9f0 2023-09-12T14:55:42.057 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2023-09-12T14:55:42.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.056+0000 7f0cda62f640 1 -- 172.21.15.131:0/1575747248 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0cc0065b20 msgr2=0x7f0cc0067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:42.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.056+0000 7f0cda62f640 1 --2- 172.21.15.131:0/1575747248 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0cc0065b20 0x7f0cc0067fe0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40cc900 tx=0x7f0cc4009040 comp rx=0 tx=0).stop 2023-09-12T14:55:42.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.056+0000 7f0cda62f640 1 -- 172.21.15.131:0/1575747248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cd415e9f0 msgr2=0x7f0cd406ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:42.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.056+0000 7f0cda62f640 1 --2- 172.21.15.131:0/1575747248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cd415e9f0 0x7f0cd406ab50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc00e970 tx=0x7f0ccc00ee30 comp rx=0 tx=0).stop 2023-09-12T14:55:42.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.056+0000 7f0cda62f640 1 -- 172.21.15.131:0/1575747248 shutdown_connections 2023-09-12T14:55:42.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.056+0000 7f0cda62f640 1 --2- 172.21.15.131:0/1575747248 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0cc0065b20 0x7f0cc0067fe0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:42.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.056+0000 7f0cda62f640 1 --2- 172.21.15.131:0/1575747248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cd415e9f0 0x7f0cd406ab50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:42.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.056+0000 7f0cda62f640 1 --2- 172.21.15.131:0/1575747248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cd40d1680 0x7f0cd41509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:42.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.056+0000 7f0cda62f640 1 --2- 172.21.15.131:0/1575747248 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cd40cf2d0 0x7f0cd4151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:42.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.056+0000 7f0cda62f640 1 -- 172.21.15.131:0/1575747248 >> 172.21.15.131:0/1575747248 conn(0x7f0cd409a200 msgr2=0x7f0cd40cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:42.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.056+0000 7f0cda62f640 1 -- 172.21.15.131:0/1575747248 shutdown_connections 2023-09-12T14:55:42.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.056+0000 7f0cda62f640 1 -- 172.21.15.131:0/1575747248 wait complete. 2023-09-12T14:55:42.076 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:55:42.076 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-12T14:55:42.083 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:55:42.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.208+0000 7fa803cad640 1 -- 172.21.15.131:0/3465286713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7e4003980 msgr2=0x7fa7e4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:42.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.208+0000 7fa803cad640 1 --2- 172.21.15.131:0/3465286713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7e4003980 0x7fa7e4003da0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40016b0 tx=0x7fa7f401c750 comp rx=0 tx=0).stop 2023-09-12T14:55:42.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.208+0000 7fa803cad640 1 -- 172.21.15.131:0/3465286713 shutdown_connections 2023-09-12T14:55:42.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.208+0000 7fa803cad640 1 --2- 172.21.15.131:0/3465286713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7e4003980 0x7fa7e4003da0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:42.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.208+0000 7fa803cad640 1 -- 172.21.15.131:0/3465286713 >> 172.21.15.131:0/3465286713 conn(0x7fa7fc069840 msgr2=0x7fa7fc0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:42.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.208+0000 7fa803cad640 1 -- 172.21.15.131:0/3465286713 shutdown_connections 2023-09-12T14:55:42.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.208+0000 7fa803cad640 1 -- 172.21.15.131:0/3465286713 wait complete. 2023-09-12T14:55:42.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.208+0000 7fa803cad640 1 Processor -- start 2023-09-12T14:55:42.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.208+0000 7fa803cad640 1 -- start start 2023-09-12T14:55:42.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa803cad640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7e4003980 0x7fa7fc151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:42.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa803cad640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7fc1e70a0 0x7fa7fc1e7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:42.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa803cad640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7fc148fa0 0x7fa7fc149420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:42.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa803cad640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc1e7bf0 con 0x7fa7fc148fa0 2023-09-12T14:55:42.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa803cad640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc149960 con 0x7fa7e4003980 2023-09-12T14:55:42.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa803cad640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc14e650 con 0x7fa7fc1e70a0 2023-09-12T14:55:42.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa80224a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7fc148fa0 0x7fa7fc149420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:42.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa80224a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7fc148fa0 0x7fa7fc149420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48792/0 (socket says 172.21.15.131:48792) 2023-09-12T14:55:42.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa80224a640 1 -- 172.21.15.131:0/4263389373 learned_addr learned my addr 172.21.15.131:0/4263389373 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:42.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa801248640 1 --2- 172.21.15.131:0/4263389373 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7fc1e70a0 0x7fa7fc1e7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:42.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa801a49640 1 --2- 172.21.15.131:0/4263389373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7e4003980 0x7fa7fc151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:42.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa80224a640 1 -- 172.21.15.131:0/4263389373 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7fc1e70a0 msgr2=0x7fa7fc1e7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:42.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa80224a640 1 --2- 172.21.15.131:0/4263389373 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7fc1e70a0 0x7fa7fc1e7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:42.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa80224a640 1 -- 172.21.15.131:0/4263389373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7e4003980 msgr2=0x7fa7fc151a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:42.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa80224a640 1 --2- 172.21.15.131:0/4263389373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7e4003980 0x7fa7fc151a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:42.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa80224a640 1 -- 172.21.15.131:0/4263389373 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7fc14e930 con 0x7fa7fc148fa0 2023-09-12T14:55:42.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa80224a640 1 --2- 172.21.15.131:0/4263389373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7fc148fa0 0x7fa7fc149420 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0020b0 tx=0x7fa7ec006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:42.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa7fa7fc640 1 -- 172.21.15.131:0/4263389373 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec00fa20 con 0x7fa7fc148fa0 2023-09-12T14:55:42.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa7fa7fc640 1 -- 172.21.15.131:0/4263389373 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa7ec00fba0 con 0x7fa7fc148fa0 2023-09-12T14:55:42.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa7fa7fc640 1 -- 172.21.15.131:0/4263389373 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec011430 con 0x7fa7fc148fa0 2023-09-12T14:55:42.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa803cad640 1 -- 172.21.15.131:0/4263389373 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7fc14ec10 con 0x7fa7fc148fa0 2023-09-12T14:55:42.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa803cad640 1 -- 172.21.15.131:0/4263389373 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7fc0af3e0 con 0x7fa7fc148fa0 2023-09-12T14:55:42.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa803cad640 1 -- 172.21.15.131:0/4263389373 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7fc073580 con 0x7fa7fc148fa0 2023-09-12T14:55:42.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa7fa7fc640 1 -- 172.21.15.131:0/4263389373 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fa7ec0117a0 con 0x7fa7fc148fa0 2023-09-12T14:55:42.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa7fa7fc640 1 --2- 172.21.15.131:0/4263389373 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa7e8065b20 0x7fa7e8067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:42.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa7fa7fc640 1 -- 172.21.15.131:0/4263389373 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa7ec083ff0 con 0x7fa7fc148fa0 2023-09-12T14:55:42.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa801a49640 1 --2- 172.21.15.131:0/4263389373 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa7e8065b20 0x7fa7e8067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:42.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.212+0000 7fa801a49640 1 --2- 172.21.15.131:0/4263389373 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa7e8065b20 0x7fa7e8067fe0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4001680 tx=0x7fa7f40194f0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:42.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.216+0000 7fa7fa7fc640 1 -- 172.21.15.131:0/4263389373 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fa7ec0518c0 con 0x7fa7fc148fa0 2023-09-12T14:55:42.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.360+0000 7fa803cad640 1 -- 172.21.15.131:0/4263389373 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7fc14e790 con 0x7fa7fc148fa0 2023-09-12T14:55:42.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.360+0000 7fa7fa7fc640 1 -- 172.21.15.131:0/4263389373 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16791 (secure 0 0 0) 0x7fa7fc14e790 con 0x7fa7fc148fa0 2023-09-12T14:55:42.365 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:42.365 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":22,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:55:39.097847+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:54:55.689983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T14:54:57.029447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:55:42.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.364+0000 7fa803cad640 1 -- 172.21.15.131:0/4263389373 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa7e8065b20 msgr2=0x7fa7e8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:42.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.364+0000 7fa803cad640 1 --2- 172.21.15.131:0/4263389373 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa7e8065b20 0x7fa7e8067fe0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4001680 tx=0x7fa7f40194f0 comp rx=0 tx=0).stop 2023-09-12T14:55:42.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.364+0000 7fa803cad640 1 -- 172.21.15.131:0/4263389373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7fc148fa0 msgr2=0x7fa7fc149420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:42.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.364+0000 7fa803cad640 1 --2- 172.21.15.131:0/4263389373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7fc148fa0 0x7fa7fc149420 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0020b0 tx=0x7fa7ec006720 comp rx=0 tx=0).stop 2023-09-12T14:55:42.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.364+0000 7fa803cad640 1 -- 172.21.15.131:0/4263389373 shutdown_connections 2023-09-12T14:55:42.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.364+0000 7fa803cad640 1 --2- 172.21.15.131:0/4263389373 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa7e8065b20 0x7fa7e8067fe0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:42.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.364+0000 7fa803cad640 1 --2- 172.21.15.131:0/4263389373 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7fc1e70a0 0x7fa7fc1e7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:42.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.364+0000 7fa803cad640 1 --2- 172.21.15.131:0/4263389373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7fc148fa0 0x7fa7fc149420 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:42.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.364+0000 7fa803cad640 1 --2- 172.21.15.131:0/4263389373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7e4003980 0x7fa7fc151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:42.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.364+0000 7fa803cad640 1 -- 172.21.15.131:0/4263389373 >> 172.21.15.131:0/4263389373 conn(0x7fa7fc069840 msgr2=0x7fa7fc15b830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:42.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.364+0000 7fa803cad640 1 -- 172.21.15.131:0/4263389373 shutdown_connections 2023-09-12T14:55:42.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.364+0000 7fa803cad640 1 -- 172.21.15.131:0/4263389373 wait complete. 2023-09-12T14:55:42.382 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T14:54:45.654004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T14:54:55.689983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T14:54:57.029447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-12T14:55:42.383 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T14:55:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d26682640 1 -- 172.21.15.131:0/2884709430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d08004d10 msgr2=0x7f1d08005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d26682640 1 --2- 172.21.15.131:0/2884709430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d08004d10 0x7f1d08005190 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800cc20 tx=0x7f1d1801df10 comp rx=0 tx=0).stop 2023-09-12T14:55:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d24c1f640 1 -- 172.21.15.131:0/2884709430 reap_dead start 2023-09-12T14:55:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d26682640 1 -- 172.21.15.131:0/2884709430 shutdown_connections 2023-09-12T14:55:42.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d26682640 1 -- 172.21.15.131:0/2884709430 >> 172.21.15.131:0/2884709430 conn(0x7f1d20069840 msgr2=0x7f1d200cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:42.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d26682640 1 -- 172.21.15.131:0/2884709430 shutdown_connections 2023-09-12T14:55:42.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d26682640 1 -- 172.21.15.131:0/2884709430 wait complete. 2023-09-12T14:55:42.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d26682640 1 Processor -- start 2023-09-12T14:55:42.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d26682640 1 -- start start 2023-09-12T14:55:42.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d26682640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d08004d10 0x7f1d20151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:42.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d26682640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d20151f30 0x7f1d2013deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:42.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d26682640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d201e4940 0x7f1d2013e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:42.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d26682640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2014d0e0 con 0x7f1d08004d10 2023-09-12T14:55:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d26682640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d201e4e40 con 0x7f1d20151f30 2023-09-12T14:55:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d26682640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d201e5120 con 0x7f1d201e4940 2023-09-12T14:55:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d1ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d08004d10 0x7f1d20151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d1ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d08004d10 0x7f1d20151580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48810/0 (socket says 172.21.15.131:48810) 2023-09-12T14:55:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d1ffff640 1 -- 172.21.15.131:0/2208821550 learned_addr learned my addr 172.21.15.131:0/2208821550 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:42.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d1f7fe640 1 --2- 172.21.15.131:0/2208821550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d20151f30 0x7f1d2013deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:42.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d1ffff640 1 -- 172.21.15.131:0/2208821550 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d201e4940 msgr2=0x7f1d2013e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:42.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d1ffff640 1 --2- 172.21.15.131:0/2208821550 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d201e4940 0x7f1d2013e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:42.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d1ffff640 1 -- 172.21.15.131:0/2208821550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d20151f30 msgr2=0x7f1d2013deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:42.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d1ffff640 1 --2- 172.21.15.131:0/2208821550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d20151f30 0x7f1d2013deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:42.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d1ffff640 1 -- 172.21.15.131:0/2208821550 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d200cbe00 con 0x7f1d08004d10 2023-09-12T14:55:42.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d1f7fe640 1 --2- 172.21.15.131:0/2208821550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d20151f30 0x7f1d2013deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:42.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d1ffff640 1 --2- 172.21.15.131:0/2208821550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d08004d10 0x7f1d20151580 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1d1000dc30 tx=0x7f1d1000b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:42.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d1cff9640 1 -- 172.21.15.131:0/2208821550 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1000bb40 con 0x7f1d08004d10 2023-09-12T14:55:42.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d1cff9640 1 -- 172.21.15.131:0/2208821550 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1d1000bca0 con 0x7f1d08004d10 2023-09-12T14:55:42.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d1cff9640 1 -- 172.21.15.131:0/2208821550 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d10003be0 con 0x7f1d08004d10 2023-09-12T14:55:42.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d26682640 1 -- 172.21.15.131:0/2208821550 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d201e5400 con 0x7f1d08004d10 2023-09-12T14:55:42.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.504+0000 7f1d26682640 1 -- 172.21.15.131:0/2208821550 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d20151ac0 con 0x7f1d08004d10 2023-09-12T14:55:42.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.508+0000 7f1d26682640 1 -- 172.21.15.131:0/2208821550 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d2014d270 con 0x7f1d08004d10 2023-09-12T14:55:42.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.508+0000 7f1d1cff9640 1 -- 172.21.15.131:0/2208821550 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f1d100050d0 con 0x7f1d08004d10 2023-09-12T14:55:42.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.508+0000 7f1d1cff9640 1 --2- 172.21.15.131:0/2208821550 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1d0c065b20 0x7f1d0c067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:42.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.508+0000 7f1d1cff9640 1 -- 172.21.15.131:0/2208821550 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f1d10083610 con 0x7f1d08004d10 2023-09-12T14:55:42.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.508+0000 7f1d1f7fe640 1 --2- 172.21.15.131:0/2208821550 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1d0c065b20 0x7f1d0c067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:42.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.508+0000 7f1d1f7fe640 1 --2- 172.21.15.131:0/2208821550 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1d0c065b20 0x7f1d0c067fe0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1d14005d90 tx=0x7f1d14005d20 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:42.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.512+0000 7f1d1cff9640 1 -- 172.21.15.131:0/2208821550 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f1d10050ec0 con 0x7f1d08004d10 2023-09-12T14:55:42.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.652+0000 7f1d26682640 1 -- 172.21.15.131:0/2208821550 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1d20073580 con 0x7f1d08004d10 2023-09-12T14:55:42.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.652+0000 7f1d1cff9640 1 -- 172.21.15.131:0/2208821550 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1d100190a0 con 0x7f1d08004d10 2023-09-12T14:55:42.657 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-09-12T14:55:42.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.656+0000 7f1d26682640 1 -- 172.21.15.131:0/2208821550 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1d0c065b20 msgr2=0x7f1d0c067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:42.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.656+0000 7f1d26682640 1 --2- 172.21.15.131:0/2208821550 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1d0c065b20 0x7f1d0c067fe0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1d14005d90 tx=0x7f1d14005d20 comp rx=0 tx=0).stop 2023-09-12T14:55:42.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.656+0000 7f1d26682640 1 -- 172.21.15.131:0/2208821550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d08004d10 msgr2=0x7f1d20151580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:42.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.656+0000 7f1d26682640 1 --2- 172.21.15.131:0/2208821550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d08004d10 0x7f1d20151580 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1d1000dc30 tx=0x7f1d1000b4b0 comp rx=0 tx=0).stop 2023-09-12T14:55:42.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.656+0000 7f1d26682640 1 -- 172.21.15.131:0/2208821550 shutdown_connections 2023-09-12T14:55:42.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.656+0000 7f1d26682640 1 --2- 172.21.15.131:0/2208821550 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1d0c065b20 0x7f1d0c067fe0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:42.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.656+0000 7f1d26682640 1 --2- 172.21.15.131:0/2208821550 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d201e4940 0x7f1d2013e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:42.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.656+0000 7f1d26682640 1 --2- 172.21.15.131:0/2208821550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d20151f30 0x7f1d2013deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:42.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.656+0000 7f1d26682640 1 --2- 172.21.15.131:0/2208821550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d08004d10 0x7f1d20151580 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:42.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.656+0000 7f1d26682640 1 -- 172.21.15.131:0/2208821550 >> 172.21.15.131:0/2208821550 conn(0x7f1d20069840 msgr2=0x7f1d200ac620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:42.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.656+0000 7f1d26682640 1 -- 172.21.15.131:0/2208821550 shutdown_connections 2023-09-12T14:55:42.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.656+0000 7f1d26682640 1 -- 172.21.15.131:0/2208821550 wait complete. 2023-09-12T14:55:42.674 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T14:55:42.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45ec9ac640 1 -- 172.21.15.131:0/576379612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45e8156930 msgr2=0x7f45e8159dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:42.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45ec9ac640 1 --2- 172.21.15.131:0/576379612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45e8156930 0x7f45e8159dc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f45d8009900 tx=0x7f45d801c850 comp rx=0 tx=0).stop 2023-09-12T14:55:42.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45ec9ac640 1 -- 172.21.15.131:0/576379612 shutdown_connections 2023-09-12T14:55:42.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45ec9ac640 1 --2- 172.21.15.131:0/576379612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45e80c9370 0x7f45e80c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:42.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45ec9ac640 1 --2- 172.21.15.131:0/576379612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45e8156930 0x7f45e8159dc0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:42.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45ec9ac640 1 -- 172.21.15.131:0/576379612 >> 172.21.15.131:0/576379612 conn(0x7f45e8069840 msgr2=0x7f45e80cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:42.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45ec9ac640 1 -- 172.21.15.131:0/576379612 shutdown_connections 2023-09-12T14:55:42.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45ec9ac640 1 -- 172.21.15.131:0/576379612 wait complete. 2023-09-12T14:55:42.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45ec9ac640 1 Processor -- start 2023-09-12T14:55:42.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45ec9ac640 1 -- start start 2023-09-12T14:55:42.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45ec9ac640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45e80abb40 0x7f45e81506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:42.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45ec9ac640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45e80c9370 0x7f45e814d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:42.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45ec9ac640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45e8156930 0x7f45e81478b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:42.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45ec9ac640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e8151fe0 con 0x7f45e80abb40 2023-09-12T14:55:42.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45ec9ac640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e8070d00 con 0x7f45e80c9370 2023-09-12T14:55:42.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45ec9ac640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e81527d0 con 0x7f45e8156930 2023-09-12T14:55:42.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45e659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45e80abb40 0x7f45e81506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:42.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45e659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45e80abb40 0x7f45e81506f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48838/0 (socket says 172.21.15.131:48838) 2023-09-12T14:55:42.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45e659c640 1 -- 172.21.15.131:0/554621179 learned_addr learned my addr 172.21.15.131:0/554621179 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:42.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45e5d9b640 1 --2- 172.21.15.131:0/554621179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45e80c9370 0x7f45e814d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:42.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45e659c640 1 -- 172.21.15.131:0/554621179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45e8156930 msgr2=0x7f45e81478b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:42.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45e659c640 1 --2- 172.21.15.131:0/554621179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45e8156930 0x7f45e81478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:42.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45e659c640 1 -- 172.21.15.131:0/554621179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45e80c9370 msgr2=0x7f45e814d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:42.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45e659c640 1 --2- 172.21.15.131:0/554621179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45e80c9370 0x7f45e814d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:42.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45e659c640 1 -- 172.21.15.131:0/554621179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d8009560 con 0x7f45e80abb40 2023-09-12T14:55:42.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45e5d9b640 1 --2- 172.21.15.131:0/554621179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45e80c9370 0x7f45e814d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:42.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45e659c640 1 --2- 172.21.15.131:0/554621179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45e80abb40 0x7f45e81506f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f45d8000c10 tx=0x7f45d80a5750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:42.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.796+0000 7f45d6ffd640 1 -- 172.21.15.131:0/554621179 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d80ba070 con 0x7f45e80abb40 2023-09-12T14:55:42.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.800+0000 7f45d6ffd640 1 -- 172.21.15.131:0/554621179 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f45d8003600 con 0x7f45e80abb40 2023-09-12T14:55:42.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.800+0000 7f45ec9ac640 1 -- 172.21.15.131:0/554621179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e8070a20 con 0x7f45e80abb40 2023-09-12T14:55:42.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.800+0000 7f45d6ffd640 1 -- 172.21.15.131:0/554621179 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d80039d0 con 0x7f45e80abb40 2023-09-12T14:55:42.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.800+0000 7f45ec9ac640 1 -- 172.21.15.131:0/554621179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e81524f0 con 0x7f45e80abb40 2023-09-12T14:55:42.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.800+0000 7f45ec9ac640 1 -- 172.21.15.131:0/554621179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e8073580 con 0x7f45e80abb40 2023-09-12T14:55:42.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.800+0000 7f45d6ffd640 1 -- 172.21.15.131:0/554621179 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f45d80b4040 con 0x7f45e80abb40 2023-09-12T14:55:42.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.800+0000 7f45d6ffd640 1 --2- 172.21.15.131:0/554621179 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f45c8065b20 0x7f45c8067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:42.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.800+0000 7f45d6ffd640 1 -- 172.21.15.131:0/554621179 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f45d811d910 con 0x7f45e80abb40 2023-09-12T14:55:42.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.800+0000 7f45e5d9b640 1 --2- 172.21.15.131:0/554621179 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f45c8065b20 0x7f45c8067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:42.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.800+0000 7f45e5d9b640 1 --2- 172.21.15.131:0/554621179 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f45c8065b20 0x7f45c8067fe0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f45e80cbd50 tx=0x7f45dc009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:42.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.804+0000 7f45d6ffd640 1 -- 172.21.15.131:0/554621179 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f45d80eb140 con 0x7f45e80abb40 2023-09-12T14:55:42.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.956+0000 7f45ec9ac640 1 -- 172.21.15.131:0/554621179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f45e80c8a00 con 0x7f45e80abb40 2023-09-12T14:55:42.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.956+0000 7f45d6ffd640 1 -- 172.21.15.131:0/554621179 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f45d80b20a0 con 0x7f45e80abb40 2023-09-12T14:55:42.959 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-09-12T14:55:42.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.956+0000 7f45ec9ac640 1 -- 172.21.15.131:0/554621179 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f45c8065b20 msgr2=0x7f45c8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:42.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.956+0000 7f45ec9ac640 1 --2- 172.21.15.131:0/554621179 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f45c8065b20 0x7f45c8067fe0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f45e80cbd50 tx=0x7f45dc009040 comp rx=0 tx=0).stop 2023-09-12T14:55:42.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.956+0000 7f45ec9ac640 1 -- 172.21.15.131:0/554621179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45e80abb40 msgr2=0x7f45e81506f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:42.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.956+0000 7f45ec9ac640 1 --2- 172.21.15.131:0/554621179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45e80abb40 0x7f45e81506f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f45d8000c10 tx=0x7f45d80a5750 comp rx=0 tx=0).stop 2023-09-12T14:55:42.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.956+0000 7f45ec9ac640 1 -- 172.21.15.131:0/554621179 shutdown_connections 2023-09-12T14:55:42.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.956+0000 7f45ec9ac640 1 --2- 172.21.15.131:0/554621179 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f45c8065b20 0x7f45c8067fe0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:42.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.956+0000 7f45ec9ac640 1 --2- 172.21.15.131:0/554621179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45e8156930 0x7f45e81478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:42.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.956+0000 7f45ec9ac640 1 --2- 172.21.15.131:0/554621179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45e80c9370 0x7f45e814d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:42.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.956+0000 7f45ec9ac640 1 --2- 172.21.15.131:0/554621179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45e80abb40 0x7f45e81506f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:42.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.956+0000 7f45ec9ac640 1 -- 172.21.15.131:0/554621179 >> 172.21.15.131:0/554621179 conn(0x7f45e8069840 msgr2=0x7f45e815b670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:42.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.956+0000 7f45ec9ac640 1 -- 172.21.15.131:0/554621179 shutdown_connections 2023-09-12T14:55:42.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:42.956+0000 7f45ec9ac640 1 -- 172.21.15.131:0/554621179 wait complete. 2023-09-12T14:55:42.977 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T14:55:43.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88c5c20640 1 -- 172.21.15.131:0/209120914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88a80047d0 msgr2=0x7f88a8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:43.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88c5c20640 1 --2- 172.21.15.131:0/209120914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88a80047d0 0x7f88a8004c50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f88c00d1ac0 tx=0x7f88b401f6e0 comp rx=0 tx=0).stop 2023-09-12T14:55:43.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88bffff640 1 -- 172.21.15.131:0/209120914 reap_dead start 2023-09-12T14:55:43.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88c5c20640 1 -- 172.21.15.131:0/209120914 shutdown_connections 2023-09-12T14:55:43.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88c5c20640 1 -- 172.21.15.131:0/209120914 >> 172.21.15.131:0/209120914 conn(0x7f88c009a200 msgr2=0x7f88c00d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:43.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88c5c20640 1 -- 172.21.15.131:0/209120914 shutdown_connections 2023-09-12T14:55:43.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88c5c20640 1 -- 172.21.15.131:0/209120914 wait complete. 2023-09-12T14:55:43.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88c5c20640 1 Processor -- start 2023-09-12T14:55:43.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88c5c20640 1 -- start start 2023-09-12T14:55:43.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88c5c20640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88a80047d0 0x7f88c009d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:43.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88c5c20640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88c0155a00 0x7f88c00afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:43.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88c5c20640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88c00b60d0 0x7f88c00b6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:43.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88c5c20640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c00b6a90 con 0x7f88a80047d0 2023-09-12T14:55:43.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88c5c20640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c00c9090 con 0x7f88c0155a00 2023-09-12T14:55:43.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88c5c20640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c00b0000 con 0x7f88c00b60d0 2023-09-12T14:55:43.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88bf7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88a80047d0 0x7f88c009d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:43.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88bf7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88a80047d0 0x7f88c009d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48848/0 (socket says 172.21.15.131:48848) 2023-09-12T14:55:43.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88bf7fe640 1 -- 172.21.15.131:0/3914139474 learned_addr learned my addr 172.21.15.131:0/3914139474 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:43.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88beffd640 1 --2- 172.21.15.131:0/3914139474 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88c0155a00 0x7f88c00afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:43.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88bffff640 1 --2- 172.21.15.131:0/3914139474 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88c00b60d0 0x7f88c00b6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:43.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88bf7fe640 1 -- 172.21.15.131:0/3914139474 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88c00b60d0 msgr2=0x7f88c00b6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:43.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88bf7fe640 1 --2- 172.21.15.131:0/3914139474 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88c00b60d0 0x7f88c00b6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:43.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88bf7fe640 1 -- 172.21.15.131:0/3914139474 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88c0155a00 msgr2=0x7f88c00afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:43.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88bf7fe640 1 --2- 172.21.15.131:0/3914139474 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88c0155a00 0x7f88c00afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:43.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88bf7fe640 1 -- 172.21.15.131:0/3914139474 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c009bf40 con 0x7f88a80047d0 2023-09-12T14:55:43.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88bffff640 1 --2- 172.21.15.131:0/3914139474 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88c00b60d0 0x7f88c00b6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:43.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88bf7fe640 1 --2- 172.21.15.131:0/3914139474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88a80047d0 0x7f88c009d560 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f88b000e990 tx=0x7f88b000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:43.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88a7fff640 1 -- 172.21.15.131:0/3914139474 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b000cbe0 con 0x7f88a80047d0 2023-09-12T14:55:43.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88a7fff640 1 -- 172.21.15.131:0/3914139474 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f88b000ce30 con 0x7f88a80047d0 2023-09-12T14:55:43.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88a7fff640 1 -- 172.21.15.131:0/3914139474 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b00116b0 con 0x7f88a80047d0 2023-09-12T14:55:43.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88c5c20640 1 -- 172.21.15.131:0/3914139474 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c009c220 con 0x7f88a80047d0 2023-09-12T14:55:43.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.100+0000 7f88c5c20640 1 -- 172.21.15.131:0/3914139474 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c0155e60 con 0x7f88a80047d0 2023-09-12T14:55:43.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.104+0000 7f88c5c20640 1 -- 172.21.15.131:0/3914139474 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c0073580 con 0x7f88a80047d0 2023-09-12T14:55:43.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.104+0000 7f88a7fff640 1 -- 172.21.15.131:0/3914139474 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f88b0011810 con 0x7f88a80047d0 2023-09-12T14:55:43.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.104+0000 7f88a7fff640 1 --2- 172.21.15.131:0/3914139474 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f88ac065b20 0x7f88ac067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:43.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.104+0000 7f88a7fff640 1 -- 172.21.15.131:0/3914139474 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f88b0084030 con 0x7f88a80047d0 2023-09-12T14:55:43.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.104+0000 7f88beffd640 1 --2- 172.21.15.131:0/3914139474 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f88ac065b20 0x7f88ac067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:43.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.104+0000 7f88beffd640 1 --2- 172.21.15.131:0/3914139474 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f88ac065b20 0x7f88ac067fe0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f88b40b0000 tx=0x7f88b401c480 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:43.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.108+0000 7f88a7fff640 1 -- 172.21.15.131:0/3914139474 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f88b0055310 con 0x7f88a80047d0 2023-09-12T14:55:43.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.248+0000 7f88c5c20640 1 -- 172.21.15.131:0/3914139474 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f88c009c410 con 0x7f88a80047d0 2023-09-12T14:55:43.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.248+0000 7f88a7fff640 1 -- 172.21.15.131:0/3914139474 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f88b001f020 con 0x7f88a80047d0 2023-09-12T14:55:43.254 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-09-12T14:55:43.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.252+0000 7f88c5c20640 1 -- 172.21.15.131:0/3914139474 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f88ac065b20 msgr2=0x7f88ac067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:43.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.252+0000 7f88c5c20640 1 --2- 172.21.15.131:0/3914139474 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f88ac065b20 0x7f88ac067fe0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f88b40b0000 tx=0x7f88b401c480 comp rx=0 tx=0).stop 2023-09-12T14:55:43.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.252+0000 7f88c5c20640 1 -- 172.21.15.131:0/3914139474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88a80047d0 msgr2=0x7f88c009d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:43.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.252+0000 7f88c5c20640 1 --2- 172.21.15.131:0/3914139474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88a80047d0 0x7f88c009d560 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f88b000e990 tx=0x7f88b000ee50 comp rx=0 tx=0).stop 2023-09-12T14:55:43.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.252+0000 7f88c5c20640 1 -- 172.21.15.131:0/3914139474 shutdown_connections 2023-09-12T14:55:43.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.252+0000 7f88c5c20640 1 --2- 172.21.15.131:0/3914139474 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f88ac065b20 0x7f88ac067fe0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:43.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.252+0000 7f88c5c20640 1 --2- 172.21.15.131:0/3914139474 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88c00b60d0 0x7f88c00b6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:43.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.252+0000 7f88c5c20640 1 --2- 172.21.15.131:0/3914139474 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88c0155a00 0x7f88c00afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:43.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.252+0000 7f88c5c20640 1 --2- 172.21.15.131:0/3914139474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88a80047d0 0x7f88c009d560 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:43.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.252+0000 7f88c5c20640 1 -- 172.21.15.131:0/3914139474 >> 172.21.15.131:0/3914139474 conn(0x7f88c009a200 msgr2=0x7f88c0157620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:43.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.252+0000 7f88c5c20640 1 -- 172.21.15.131:0/3914139474 shutdown_connections 2023-09-12T14:55:43.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.252+0000 7f88c5c20640 1 -- 172.21.15.131:0/3914139474 wait complete. 2023-09-12T14:55:43.271 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-12T14:55:43.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.388+0000 7fb6d6a39640 1 -- 172.21.15.131:0/4004318986 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6d0156930 msgr2=0x7fb6d0159dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:43.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.388+0000 7fb6d6a39640 1 --2- 172.21.15.131:0/4004318986 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6d0156930 0x7fb6d0159dc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0009c80 tx=0x7fb6c001cd50 comp rx=0 tx=0).stop 2023-09-12T14:55:43.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.388+0000 7fb6d6a39640 1 -- 172.21.15.131:0/4004318986 shutdown_connections 2023-09-12T14:55:43.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.388+0000 7fb6d6a39640 1 --2- 172.21.15.131:0/4004318986 >> v2:172.21.15.187:3300/0 conn(0x7fb6d00abb40 0x7fb6d006b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:43.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.388+0000 7fb6d6a39640 1 --2- 172.21.15.131:0/4004318986 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6d0156930 0x7fb6d0159dc0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:43.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.388+0000 7fb6d6a39640 1 -- 172.21.15.131:0/4004318986 >> 172.21.15.131:0/4004318986 conn(0x7fb6d0069840 msgr2=0x7fb6d00cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:43.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.388+0000 7fb6d6a39640 1 -- 172.21.15.131:0/4004318986 shutdown_connections 2023-09-12T14:55:43.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.388+0000 7fb6d6a39640 1 -- 172.21.15.131:0/4004318986 wait complete. 2023-09-12T14:55:43.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.388+0000 7fb6d6a39640 1 Processor -- start 2023-09-12T14:55:43.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.388+0000 7fb6d6a39640 1 -- start start 2023-09-12T14:55:43.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6d6a39640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6d00abb40 0x7fb6d0152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:43.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6d6a39640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6d00c9370 0x7fb6d014dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:43.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6d6a39640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6d0156930 0x7fb6d00cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:43.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6d6a39640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d014e280 con 0x7fb6d00abb40 2023-09-12T14:55:43.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6d6a39640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d01499b0 con 0x7fb6d0156930 2023-09-12T14:55:43.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6d6a39640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d0154b00 con 0x7fb6d00c9370 2023-09-12T14:55:43.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6cffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6d00abb40 0x7fb6d0152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:43.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6cffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6d00abb40 0x7fb6d0152e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48860/0 (socket says 172.21.15.131:48860) 2023-09-12T14:55:43.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6cffff640 1 -- 172.21.15.131:0/1215040801 learned_addr learned my addr 172.21.15.131:0/1215040801 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:43.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6d4fd6640 1 --2- 172.21.15.131:0/1215040801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6d0156930 0x7fb6d00cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:43.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6cf7fe640 1 --2- 172.21.15.131:0/1215040801 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6d00c9370 0x7fb6d014dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:43.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6cffff640 1 -- 172.21.15.131:0/1215040801 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6d00c9370 msgr2=0x7fb6d014dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:43.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6cffff640 1 --2- 172.21.15.131:0/1215040801 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6d00c9370 0x7fb6d014dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:43.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6cffff640 1 -- 172.21.15.131:0/1215040801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6d0156930 msgr2=0x7fb6d00cb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:43.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6cffff640 1 --2- 172.21.15.131:0/1215040801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6d0156930 0x7fb6d00cb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:43.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6cffff640 1 -- 172.21.15.131:0/1215040801 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b80094a0 con 0x7fb6d00abb40 2023-09-12T14:55:43.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6cf7fe640 1 --2- 172.21.15.131:0/1215040801 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6d00c9370 0x7fb6d014dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:43.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6cffff640 1 --2- 172.21.15.131:0/1215040801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6d00abb40 0x7fb6d0152e00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0000f90 tx=0x7fb6c00092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:43.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6ccff9640 1 -- 172.21.15.131:0/1215040801 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c00aeb40 con 0x7fb6d00abb40 2023-09-12T14:55:43.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6ccff9640 1 -- 172.21.15.131:0/1215040801 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb6c00aed90 con 0x7fb6d00abb40 2023-09-12T14:55:43.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6d6a39640 1 -- 172.21.15.131:0/1215040801 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c00098e0 con 0x7fb6d00abb40 2023-09-12T14:55:43.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6ccff9640 1 -- 172.21.15.131:0/1215040801 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0004240 con 0x7fb6d00abb40 2023-09-12T14:55:43.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6d6a39640 1 -- 172.21.15.131:0/1215040801 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d014f1b0 con 0x7fb6d00abb40 2023-09-12T14:55:43.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6d6a39640 1 -- 172.21.15.131:0/1215040801 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d0154960 con 0x7fb6d00abb40 2023-09-12T14:55:43.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6ccff9640 1 -- 172.21.15.131:0/1215040801 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fb6c00b4040 con 0x7fb6d00abb40 2023-09-12T14:55:43.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6ccff9640 1 --2- 172.21.15.131:0/1215040801 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb6b0065b50 0x7fb6b0068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:43.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6cf7fe640 1 --2- 172.21.15.131:0/1215040801 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb6b0065b50 0x7fb6b0068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:43.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6ccff9640 1 -- 172.21.15.131:0/1215040801 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fb6c011eaf0 con 0x7fb6d00abb40 2023-09-12T14:55:43.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.392+0000 7fb6cf7fe640 1 --2- 172.21.15.131:0/1215040801 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb6b0065b50 0x7fb6b0068010 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb6d015d430 tx=0x7fb6b80091e0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:43.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.396+0000 7fb6ccff9640 1 -- 172.21.15.131:0/1215040801 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fb6d0154960 con 0x7fb6d00abb40 2023-09-12T14:55:43.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.548+0000 7fb6d6a39640 1 -- 172.21.15.131:0/1215040801 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb6d0154960 con 0x7fb6d00abb40 2023-09-12T14:55:43.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.548+0000 7fb6ccff9640 1 -- 172.21.15.131:0/1215040801 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb6d0154960 con 0x7fb6d00abb40 2023-09-12T14:55:43.553 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2023-09-12T14:55:43.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.552+0000 7fb6d6a39640 1 -- 172.21.15.131:0/1215040801 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb6b0065b50 msgr2=0x7fb6b0068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:43.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.552+0000 7fb6d6a39640 1 --2- 172.21.15.131:0/1215040801 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb6b0065b50 0x7fb6b0068010 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb6d015d430 tx=0x7fb6b80091e0 comp rx=0 tx=0).stop 2023-09-12T14:55:43.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.552+0000 7fb6d6a39640 1 -- 172.21.15.131:0/1215040801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6d00abb40 msgr2=0x7fb6d0152e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:43.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.552+0000 7fb6d6a39640 1 --2- 172.21.15.131:0/1215040801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6d00abb40 0x7fb6d0152e00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0000f90 tx=0x7fb6c00092c0 comp rx=0 tx=0).stop 2023-09-12T14:55:43.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.552+0000 7fb6d6a39640 1 -- 172.21.15.131:0/1215040801 shutdown_connections 2023-09-12T14:55:43.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.552+0000 7fb6d6a39640 1 --2- 172.21.15.131:0/1215040801 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb6b0065b50 0x7fb6b0068010 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:43.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.552+0000 7fb6d6a39640 1 --2- 172.21.15.131:0/1215040801 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6d00c9370 0x7fb6d014dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:43.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.552+0000 7fb6d6a39640 1 --2- 172.21.15.131:0/1215040801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6d0156930 0x7fb6d00cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:43.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.552+0000 7fb6d6a39640 1 --2- 172.21.15.131:0/1215040801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6d00abb40 0x7fb6d0152e00 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:43.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.552+0000 7fb6d6a39640 1 -- 172.21.15.131:0/1215040801 >> 172.21.15.131:0/1215040801 conn(0x7fb6d0069840 msgr2=0x7fb6d015b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:43.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.552+0000 7fb6d6a39640 1 -- 172.21.15.131:0/1215040801 shutdown_connections 2023-09-12T14:55:43.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.552+0000 7fb6d6a39640 1 -- 172.21.15.131:0/1215040801 wait complete. 2023-09-12T14:55:43.570 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi187.front.sepia.ceph.com... 2023-09-12T14:55:43.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:43.571 DEBUG:teuthology.orchestra.run.smithi187:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-09-12T14:55:43.586 INFO:teuthology.orchestra.run:waiting for 300 2023-09-12T14:55:43.587 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2023-09-12T14:55:43.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:55:43.588 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-09-12T14:55:43.635 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-09-12T14:55:43.636 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T14:55:43.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fcce541f640 1 -- 172.21.15.131:0/550734647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce006f390 msgr2=0x7fcce006f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:43.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fcce541f640 1 --2- 172.21.15.131:0/550734647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce006f390 0x7fcce006f820 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcce006c710 tx=0x7fccd801cd30 comp rx=0 tx=0).stop 2023-09-12T14:55:43.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fcce541f640 1 -- 172.21.15.131:0/550734647 shutdown_connections 2023-09-12T14:55:43.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fcce541f640 1 --2- 172.21.15.131:0/550734647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce006f390 0x7fcce006f820 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:43.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fcce541f640 1 --2- 172.21.15.131:0/550734647 >> v2:172.21.15.187:3300/0 conn(0x7fcce0158180 0x7fcce00cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:43.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fcce541f640 1 --2- 172.21.15.131:0/550734647 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcce00ab730 0x7fcce00abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:43.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fcce541f640 1 -- 172.21.15.131:0/550734647 >> 172.21.15.131:0/550734647 conn(0x7fcce0069840 msgr2=0x7fcce00cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:43.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fcce541f640 1 -- 172.21.15.131:0/550734647 shutdown_connections 2023-09-12T14:55:43.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fcce541f640 1 -- 172.21.15.131:0/550734647 wait complete. 2023-09-12T14:55:43.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fcce541f640 1 Processor -- start 2023-09-12T14:55:43.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fcce541f640 1 -- start start 2023-09-12T14:55:43.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fcce541f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcce006f390 0x7fcce014b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:43.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fcce541f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcce00ab730 0x7fcce00aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:43.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fcce541f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce0158180 0x7fcce00aae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:43.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fcce541f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce01f84e0 con 0x7fcce0158180 2023-09-12T14:55:43.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fcce541f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce01f7720 con 0x7fcce006f390 2023-09-12T14:55:43.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fcce541f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce01f7a00 con 0x7fcce00ab730 2023-09-12T14:55:43.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fccdf7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce0158180 0x7fcce00aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:43.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fccdf7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce0158180 0x7fcce00aae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48884/0 (socket says 172.21.15.131:48884) 2023-09-12T14:55:43.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fccdf7fe640 1 -- 172.21.15.131:0/122601737 learned_addr learned my addr 172.21.15.131:0/122601737 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:43.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fccdeffd640 1 --2- 172.21.15.131:0/122601737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcce006f390 0x7fcce014b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:43.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fccde7fc640 1 --2- 172.21.15.131:0/122601737 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcce00ab730 0x7fcce00aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:43.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fccdf7fe640 1 -- 172.21.15.131:0/122601737 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcce00ab730 msgr2=0x7fcce00aa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:43.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fccdf7fe640 1 --2- 172.21.15.131:0/122601737 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcce00ab730 0x7fcce00aa8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:43.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fccdf7fe640 1 -- 172.21.15.131:0/122601737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcce006f390 msgr2=0x7fcce014b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:43.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fccdf7fe640 1 --2- 172.21.15.131:0/122601737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcce006f390 0x7fcce014b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:43.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fccdf7fe640 1 -- 172.21.15.131:0/122601737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd80098e0 con 0x7fcce0158180 2023-09-12T14:55:43.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.756+0000 7fccdf7fe640 1 --2- 172.21.15.131:0/122601737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce0158180 0x7fcce00aae20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009bd0 tx=0x7fccd8003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:43.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.760+0000 7fccc77fe640 1 -- 172.21.15.131:0/122601737 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8004e60 con 0x7fcce0158180 2023-09-12T14:55:43.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.760+0000 7fccc77fe640 1 -- 172.21.15.131:0/122601737 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fccd80ad790 con 0x7fcce0158180 2023-09-12T14:55:43.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.760+0000 7fcce541f640 1 -- 172.21.15.131:0/122601737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce01f7ce0 con 0x7fcce0158180 2023-09-12T14:55:43.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.760+0000 7fccc77fe640 1 -- 172.21.15.131:0/122601737 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd80adb80 con 0x7fcce0158180 2023-09-12T14:55:43.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.760+0000 7fcce541f640 1 -- 172.21.15.131:0/122601737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce01f4f00 con 0x7fcce0158180 2023-09-12T14:55:43.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.760+0000 7fcce541f640 1 -- 172.21.15.131:0/122601737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcce0073580 con 0x7fcce0158180 2023-09-12T14:55:43.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.760+0000 7fccc77fe640 1 -- 172.21.15.131:0/122601737 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fccd80b4040 con 0x7fcce0158180 2023-09-12T14:55:43.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.760+0000 7fccc77fe640 1 --2- 172.21.15.131:0/122601737 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fcccc065b20 0x7fcccc067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:43.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.760+0000 7fccdeffd640 1 --2- 172.21.15.131:0/122601737 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fcccc065b20 0x7fcccc067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:43.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.760+0000 7fccc77fe640 1 -- 172.21.15.131:0/122601737 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fccd811d990 con 0x7fcce0158180 2023-09-12T14:55:43.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.760+0000 7fccdeffd640 1 --2- 172.21.15.131:0/122601737 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fcccc065b20 0x7fcccc067fe0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fccd0005950 tx=0x7fccd0005db0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:43.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.764+0000 7fccc77fe640 1 -- 172.21.15.131:0/122601737 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fccd80eb1c0 con 0x7fcce0158180 2023-09-12T14:55:43.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.944+0000 7fcce541f640 1 -- 172.21.15.131:0/122601737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcce01525b0 con 0x7fcce0158180 2023-09-12T14:55:43.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.944+0000 7fccc77fe640 1 -- 172.21.15.131:0/122601737 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4907 (secure 0 0 0) 0x7fccd80b2020 con 0x7fcce0158180 2023-09-12T14:55:43.948 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:43.948 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4344,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/491327908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":491327908},{"type":"v1","addr":"172.21.15.131:6837","nonce":491327908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3470212105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3470212105},{"type":"v1","addr":"172.21.15.187:6833","nonce":3470212105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/91478708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":91478708},{"type":"v1","addr":"172.21.15.131:6836","nonce":91478708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-12T14:54:58.030444+0000","modified":"2023-09-12T14:55:00.236338+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4396":{"gid":4396,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/290727181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":290727181},{"type":"v1","addr":"172.21.15.187:6835","nonce":290727181}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-12T14:55:43.948 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 6 2023-09-12T14:55:43.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.948+0000 7fcce541f640 1 -- 172.21.15.131:0/122601737 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fcccc065b20 msgr2=0x7fcccc067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:43.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.948+0000 7fcce541f640 1 --2- 172.21.15.131:0/122601737 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fcccc065b20 0x7fcccc067fe0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fccd0005950 tx=0x7fccd0005db0 comp rx=0 tx=0).stop 2023-09-12T14:55:43.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.948+0000 7fcce541f640 1 -- 172.21.15.131:0/122601737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce0158180 msgr2=0x7fcce00aae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:43.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.948+0000 7fcce541f640 1 --2- 172.21.15.131:0/122601737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce0158180 0x7fcce00aae20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009bd0 tx=0x7fccd8003c40 comp rx=0 tx=0).stop 2023-09-12T14:55:43.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.948+0000 7fcce541f640 1 -- 172.21.15.131:0/122601737 shutdown_connections 2023-09-12T14:55:43.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.948+0000 7fcce541f640 1 --2- 172.21.15.131:0/122601737 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fcccc065b20 0x7fcccc067fe0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:43.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.948+0000 7fcce541f640 1 --2- 172.21.15.131:0/122601737 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcce00ab730 0x7fcce00aa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:43.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.948+0000 7fcce541f640 1 --2- 172.21.15.131:0/122601737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce0158180 0x7fcce00aae20 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:43.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.948+0000 7fcce541f640 1 --2- 172.21.15.131:0/122601737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcce006f390 0x7fcce014b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:43.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.948+0000 7fcce541f640 1 -- 172.21.15.131:0/122601737 >> 172.21.15.131:0/122601737 conn(0x7fcce0069840 msgr2=0x7fcce0072320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:43.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.948+0000 7fcce541f640 1 -- 172.21.15.131:0/122601737 shutdown_connections 2023-09-12T14:55:43.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:43.948+0000 7fcce541f640 1 -- 172.21.15.131:0/122601737 wait complete. 2023-09-12T14:55:43.965 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:55:44.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.084+0000 7fec857a9640 1 -- 172.21.15.131:0/1786068860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec800b2270 msgr2=0x7fec800cf240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:44.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.084+0000 7fec857a9640 1 --2- 172.21.15.131:0/1786068860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec800b2270 0x7fec800cf240 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fec74009900 tx=0x7fec7401c800 comp rx=0 tx=0).stop 2023-09-12T14:55:44.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.084+0000 7fec857a9640 1 -- 172.21.15.131:0/1786068860 shutdown_connections 2023-09-12T14:55:44.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.084+0000 7fec857a9640 1 --2- 172.21.15.131:0/1786068860 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec800cb790 0x7fec800cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.084+0000 7fec857a9640 1 --2- 172.21.15.131:0/1786068860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec800b2270 0x7fec800cf240 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.084+0000 7fec857a9640 1 -- 172.21.15.131:0/1786068860 >> 172.21.15.131:0/1786068860 conn(0x7fec8009a210 msgr2=0x7fec800d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:44.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.084+0000 7fec857a9640 1 -- 172.21.15.131:0/1786068860 shutdown_connections 2023-09-12T14:55:44.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.084+0000 7fec857a9640 1 -- 172.21.15.131:0/1786068860 wait complete. 2023-09-12T14:55:44.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.088+0000 7fec857a9640 1 Processor -- start 2023-09-12T14:55:44.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.088+0000 7fec857a9640 1 -- start start 2023-09-12T14:55:44.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.088+0000 7fec857a9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec800b2270 0x7fec801554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:44.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.088+0000 7fec857a9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec800cb790 0x7fec800c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:44.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.088+0000 7fec857a9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec8015f3f0 0x7fec8015ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:44.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.088+0000 7fec857a9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec800b4480 con 0x7fec800b2270 2023-09-12T14:55:44.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.088+0000 7fec857a9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec800b4d20 con 0x7fec8015f3f0 2023-09-12T14:55:44.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.088+0000 7fec857a9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8015d500 con 0x7fec800cb790 2023-09-12T14:55:44.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.088+0000 7fec7effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec800b2270 0x7fec801554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:44.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.088+0000 7fec7effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec800b2270 0x7fec801554e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48914/0 (socket says 172.21.15.131:48914) 2023-09-12T14:55:44.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.088+0000 7fec7effd640 1 -- 172.21.15.131:0/1384674017 learned_addr learned my addr 172.21.15.131:0/1384674017 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:44.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.088+0000 7fec7effd640 1 -- 172.21.15.131:0/1384674017 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec800cb790 msgr2=0x7fec800c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:44.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.088+0000 7fec7effd640 1 --2- 172.21.15.131:0/1384674017 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec800cb790 0x7fec800c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.088+0000 7fec7effd640 1 -- 172.21.15.131:0/1384674017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec8015f3f0 msgr2=0x7fec8015ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:44.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.088+0000 7fec7f7fe640 1 --2- 172.21.15.131:0/1384674017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec8015f3f0 0x7fec8015ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:44.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.088+0000 7fec7effd640 1 --2- 172.21.15.131:0/1384674017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec8015f3f0 0x7fec8015ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.088+0000 7fec7effd640 1 -- 172.21.15.131:0/1384674017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec74009560 con 0x7fec800b2270 2023-09-12T14:55:44.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.088+0000 7fec7effd640 1 --2- 172.21.15.131:0/1384674017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec800b2270 0x7fec801554e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fec7000c3a0 tx=0x7fec7000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:44.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.088+0000 7fec677fe640 1 -- 172.21.15.131:0/1384674017 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec700052d0 con 0x7fec800b2270 2023-09-12T14:55:44.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.088+0000 7fec677fe640 1 -- 172.21.15.131:0/1384674017 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fec70007020 con 0x7fec800b2270 2023-09-12T14:55:44.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.088+0000 7fec677fe640 1 -- 172.21.15.131:0/1384674017 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec70007410 con 0x7fec800b2270 2023-09-12T14:55:44.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.088+0000 7fec857a9640 1 -- 172.21.15.131:0/1384674017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec800b46d0 con 0x7fec800b2270 2023-09-12T14:55:44.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.088+0000 7fec857a9640 1 -- 172.21.15.131:0/1384674017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec80157b20 con 0x7fec800b2270 2023-09-12T14:55:44.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.088+0000 7fec857a9640 1 -- 172.21.15.131:0/1384674017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec80073580 con 0x7fec800b2270 2023-09-12T14:55:44.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.092+0000 7fec677fe640 1 -- 172.21.15.131:0/1384674017 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fec70007570 con 0x7fec800b2270 2023-09-12T14:55:44.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.092+0000 7fec677fe640 1 --2- 172.21.15.131:0/1384674017 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fec6c065b20 0x7fec6c067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:44.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.092+0000 7fec677fe640 1 -- 172.21.15.131:0/1384674017 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fec700829e0 con 0x7fec800b2270 2023-09-12T14:55:44.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.092+0000 7fec7e7fc640 1 --2- 172.21.15.131:0/1384674017 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fec6c065b20 0x7fec6c067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:44.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.092+0000 7fec7e7fc640 1 --2- 172.21.15.131:0/1384674017 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fec6c065b20 0x7fec6c067fe0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fec740af000 tx=0x7fec740195a0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:44.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.096+0000 7fec677fe640 1 -- 172.21.15.131:0/1384674017 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fec700502a0 con 0x7fec800b2270 2023-09-12T14:55:44.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.240+0000 7fec857a9640 1 -- 172.21.15.131:0/1384674017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec8009a730 con 0x7fec800b2270 2023-09-12T14:55:44.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.244+0000 7fec677fe640 1 -- 172.21.15.131:0/1384674017 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16791 (secure 0 0 0) 0x7fec700538c0 con 0x7fec800b2270 2023-09-12T14:55:44.247 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:44.247 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":22,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:55:39.097847+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:54:55.689983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T14:54:57.029447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:55:44.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.244+0000 7fec857a9640 1 -- 172.21.15.131:0/1384674017 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fec6c065b20 msgr2=0x7fec6c067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:44.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.244+0000 7fec857a9640 1 --2- 172.21.15.131:0/1384674017 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fec6c065b20 0x7fec6c067fe0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fec740af000 tx=0x7fec740195a0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.244+0000 7fec857a9640 1 -- 172.21.15.131:0/1384674017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec800b2270 msgr2=0x7fec801554e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:44.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.244+0000 7fec857a9640 1 --2- 172.21.15.131:0/1384674017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec800b2270 0x7fec801554e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fec7000c3a0 tx=0x7fec7000c860 comp rx=0 tx=0).stop 2023-09-12T14:55:44.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.244+0000 7fec857a9640 1 -- 172.21.15.131:0/1384674017 shutdown_connections 2023-09-12T14:55:44.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.244+0000 7fec857a9640 1 --2- 172.21.15.131:0/1384674017 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fec6c065b20 0x7fec6c067fe0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.244+0000 7fec857a9640 1 --2- 172.21.15.131:0/1384674017 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec800cb790 0x7fec800c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.244+0000 7fec857a9640 1 --2- 172.21.15.131:0/1384674017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec8015f3f0 0x7fec8015ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.244+0000 7fec857a9640 1 --2- 172.21.15.131:0/1384674017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec800b2270 0x7fec801554e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.244+0000 7fec857a9640 1 -- 172.21.15.131:0/1384674017 >> 172.21.15.131:0/1384674017 conn(0x7fec8009a210 msgr2=0x7fec800cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:44.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.244+0000 7fec857a9640 1 -- 172.21.15.131:0/1384674017 shutdown_connections 2023-09-12T14:55:44.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.244+0000 7fec857a9640 1 -- 172.21.15.131:0/1384674017 wait complete. 2023-09-12T14:55:44.264 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T14:54:45.654004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T14:54:55.689983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T14:54:57.029447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-12T14:55:44.265 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T14:55:44.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.384+0000 7fd98bd4b640 1 -- 172.21.15.131:0/3083243682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd970004d10 msgr2=0x7fd970005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:44.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.384+0000 7fd98bd4b640 1 --2- 172.21.15.131:0/3083243682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd970004d10 0x7fd970005190 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd97c00c640 tx=0x7fd97c01d320 comp rx=0 tx=0).stop 2023-09-12T14:55:44.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.384+0000 7fd98a2e8640 1 -- 172.21.15.131:0/3083243682 reap_dead start 2023-09-12T14:55:44.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.384+0000 7fd98bd4b640 1 -- 172.21.15.131:0/3083243682 shutdown_connections 2023-09-12T14:55:44.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.384+0000 7fd98bd4b640 1 -- 172.21.15.131:0/3083243682 >> 172.21.15.131:0/3083243682 conn(0x7fd9840ceac0 msgr2=0x7fd9840ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:44.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.384+0000 7fd98bd4b640 1 -- 172.21.15.131:0/3083243682 shutdown_connections 2023-09-12T14:55:44.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.384+0000 7fd98bd4b640 1 -- 172.21.15.131:0/3083243682 wait complete. 2023-09-12T14:55:44.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.384+0000 7fd98bd4b640 1 Processor -- start 2023-09-12T14:55:44.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.384+0000 7fd98bd4b640 1 -- start start 2023-09-12T14:55:44.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd98bd4b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd970004d10 0x7fd9840b4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:44.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd98bd4b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9840abee0 0x7fd9840ac360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:44.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd98bd4b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9840c6be0 0x7fd9840c7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:44.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd98bd4b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9840ac8a0 con 0x7fd9840abee0 2023-09-12T14:55:44.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd98bd4b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9840b4190 con 0x7fd9840c6be0 2023-09-12T14:55:44.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd98bd4b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9840b3580 con 0x7fd970004d10 2023-09-12T14:55:44.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd9892e6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9840abee0 0x7fd9840ac360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:44.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd9892e6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9840abee0 0x7fd9840ac360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48942/0 (socket says 172.21.15.131:48942) 2023-09-12T14:55:44.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd9892e6640 1 -- 172.21.15.131:0/498645555 learned_addr learned my addr 172.21.15.131:0/498645555 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:44.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd989ae7640 1 --2- 172.21.15.131:0/498645555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd970004d10 0x7fd9840b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:44.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd98a2e8640 1 --2- 172.21.15.131:0/498645555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9840c6be0 0x7fd9840c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:44.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd9892e6640 1 -- 172.21.15.131:0/498645555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd970004d10 msgr2=0x7fd9840b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:44.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd9892e6640 1 --2- 172.21.15.131:0/498645555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd970004d10 0x7fd9840b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd9892e6640 1 -- 172.21.15.131:0/498645555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9840c6be0 msgr2=0x7fd9840c7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:44.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd9892e6640 1 --2- 172.21.15.131:0/498645555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9840c6be0 0x7fd9840c7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd9892e6640 1 -- 172.21.15.131:0/498645555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9840b3860 con 0x7fd9840abee0 2023-09-12T14:55:44.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd9892e6640 1 --2- 172.21.15.131:0/498645555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9840abee0 0x7fd9840ac360 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd97400e990 tx=0x7fd97400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:44.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd97a7fc640 1 -- 172.21.15.131:0/498645555 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97400cbe0 con 0x7fd9840abee0 2023-09-12T14:55:44.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd97a7fc640 1 -- 172.21.15.131:0/498645555 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd97400ce30 con 0x7fd9840abee0 2023-09-12T14:55:44.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd98bd4b640 1 -- 172.21.15.131:0/498645555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9840b3b40 con 0x7fd9840abee0 2023-09-12T14:55:44.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd97a7fc640 1 -- 172.21.15.131:0/498645555 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd974005280 con 0x7fd9840abee0 2023-09-12T14:55:44.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd98bd4b640 1 -- 172.21.15.131:0/498645555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9840b31e0 con 0x7fd9840abee0 2023-09-12T14:55:44.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd98bd4b640 1 -- 172.21.15.131:0/498645555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9840b4320 con 0x7fd9840abee0 2023-09-12T14:55:44.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd97a7fc640 1 -- 172.21.15.131:0/498645555 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fd9740053e0 con 0x7fd9840abee0 2023-09-12T14:55:44.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd97a7fc640 1 --2- 172.21.15.131:0/498645555 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd968065b50 0x7fd968068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:44.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd97a7fc640 1 -- 172.21.15.131:0/498645555 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fd974084100 con 0x7fd9840abee0 2023-09-12T14:55:44.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd989ae7640 1 --2- 172.21.15.131:0/498645555 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd968065b50 0x7fd968068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:44.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.388+0000 7fd989ae7640 1 --2- 172.21.15.131:0/498645555 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd968065b50 0x7fd968068010 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd9800058d0 tx=0x7fd980005860 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:44.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.392+0000 7fd97a7fc640 1 -- 172.21.15.131:0/498645555 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fd974051a20 con 0x7fd9840abee0 2023-09-12T14:55:44.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.548+0000 7fd98bd4b640 1 -- 172.21.15.131:0/498645555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd9840aba20 con 0x7fd9840abee0 2023-09-12T14:55:44.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.548+0000 7fd97a7fc640 1 -- 172.21.15.131:0/498645555 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd97401f0a0 con 0x7fd9840abee0 2023-09-12T14:55:44.550 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-09-12T14:55:44.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.548+0000 7fd98bd4b640 1 -- 172.21.15.131:0/498645555 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd968065b50 msgr2=0x7fd968068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:44.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.548+0000 7fd98bd4b640 1 --2- 172.21.15.131:0/498645555 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd968065b50 0x7fd968068010 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd9800058d0 tx=0x7fd980005860 comp rx=0 tx=0).stop 2023-09-12T14:55:44.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.548+0000 7fd98bd4b640 1 -- 172.21.15.131:0/498645555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9840abee0 msgr2=0x7fd9840ac360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:44.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.548+0000 7fd98bd4b640 1 --2- 172.21.15.131:0/498645555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9840abee0 0x7fd9840ac360 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd97400e990 tx=0x7fd97400ee50 comp rx=0 tx=0).stop 2023-09-12T14:55:44.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.548+0000 7fd98bd4b640 1 -- 172.21.15.131:0/498645555 shutdown_connections 2023-09-12T14:55:44.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.548+0000 7fd98bd4b640 1 --2- 172.21.15.131:0/498645555 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd968065b50 0x7fd968068010 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.548+0000 7fd98bd4b640 1 --2- 172.21.15.131:0/498645555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9840c6be0 0x7fd9840c7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.548+0000 7fd98bd4b640 1 --2- 172.21.15.131:0/498645555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9840abee0 0x7fd9840ac360 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.548+0000 7fd98bd4b640 1 --2- 172.21.15.131:0/498645555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd970004d10 0x7fd9840b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.548+0000 7fd98bd4b640 1 -- 172.21.15.131:0/498645555 >> 172.21.15.131:0/498645555 conn(0x7fd9840ceac0 msgr2=0x7fd98415a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:44.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.548+0000 7fd98bd4b640 1 -- 172.21.15.131:0/498645555 shutdown_connections 2023-09-12T14:55:44.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.548+0000 7fd98bd4b640 1 -- 172.21.15.131:0/498645555 wait complete. 2023-09-12T14:55:44.568 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T14:55:44.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.680+0000 7fc7e9c31640 1 -- 172.21.15.131:0/1385507704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e40aeb90 msgr2=0x7fc7e40af000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:44.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.680+0000 7fc7e9c31640 1 --2- 172.21.15.131:0/1385507704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e40aeb90 0x7fc7e40af000 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4070bf0 tx=0x7fc7dc01c7e0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.680+0000 7fc7e9c31640 1 -- 172.21.15.131:0/1385507704 shutdown_connections 2023-09-12T14:55:44.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.680+0000 7fc7e9c31640 1 --2- 172.21.15.131:0/1385507704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e40aeb90 0x7fc7e40af000 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.680+0000 7fc7e9c31640 1 --2- 172.21.15.131:0/1385507704 >> v2:172.21.15.187:3300/0 conn(0x7fc7e40cd0c0 0x7fc7e40cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.680+0000 7fc7e9c31640 1 -- 172.21.15.131:0/1385507704 >> 172.21.15.131:0/1385507704 conn(0x7fc7e40ceae0 msgr2=0x7fc7e40ceef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:44.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.680+0000 7fc7e9c31640 1 -- 172.21.15.131:0/1385507704 shutdown_connections 2023-09-12T14:55:44.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.680+0000 7fc7e9c31640 1 -- 172.21.15.131:0/1385507704 wait complete. 2023-09-12T14:55:44.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.680+0000 7fc7e9c31640 1 Processor -- start 2023-09-12T14:55:44.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.680+0000 7fc7e9c31640 1 -- start start 2023-09-12T14:55:44.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.680+0000 7fc7e9c31640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7e40aeb90 0x7fc7e40b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:44.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.680+0000 7fc7e9c31640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7e40cd0c0 0x7fc7e41594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:44.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.680+0000 7fc7e9c31640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e40cd4a0 0x7fc7e4147920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:44.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.680+0000 7fc7e9c31640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e4158240 con 0x7fc7e40cd4a0 2023-09-12T14:55:44.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.680+0000 7fc7e9c31640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e4147e60 con 0x7fc7e40cd0c0 2023-09-12T14:55:44.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.680+0000 7fc7e9c31640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e4157c80 con 0x7fc7e40aeb90 2023-09-12T14:55:44.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.680+0000 7fc7e3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e40cd4a0 0x7fc7e4147920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:44.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.680+0000 7fc7e3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e40cd4a0 0x7fc7e4147920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48958/0 (socket says 172.21.15.131:48958) 2023-09-12T14:55:44.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.680+0000 7fc7e3fff640 1 -- 172.21.15.131:0/3807407298 learned_addr learned my addr 172.21.15.131:0/3807407298 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:44.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.680+0000 7fc7e3fff640 1 -- 172.21.15.131:0/3807407298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7e40aeb90 msgr2=0x7fc7e40b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:44.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.680+0000 7fc7e3fff640 1 --2- 172.21.15.131:0/3807407298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7e40aeb90 0x7fc7e40b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.680+0000 7fc7e3fff640 1 -- 172.21.15.131:0/3807407298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7e40cd0c0 msgr2=0x7fc7e41594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:44.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.680+0000 7fc7e2ffd640 1 --2- 172.21.15.131:0/3807407298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7e40cd0c0 0x7fc7e41594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:44.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.680+0000 7fc7e3fff640 1 --2- 172.21.15.131:0/3807407298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7e40cd0c0 0x7fc7e41594c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.680+0000 7fc7e3fff640 1 -- 172.21.15.131:0/3807407298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7dc009560 con 0x7fc7e40cd4a0 2023-09-12T14:55:44.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.680+0000 7fc7e3fff640 1 --2- 172.21.15.131:0/3807407298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e40cd4a0 0x7fc7e4147920 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc0193f0 tx=0x7fc7dc0a5640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:44.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.684+0000 7fc7dbfff640 1 -- 172.21.15.131:0/3807407298 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc003800 con 0x7fc7e40cd4a0 2023-09-12T14:55:44.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.684+0000 7fc7dbfff640 1 -- 172.21.15.131:0/3807407298 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc7dc0a5830 con 0x7fc7e40cd4a0 2023-09-12T14:55:44.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.684+0000 7fc7e9c31640 1 -- 172.21.15.131:0/3807407298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7e4157f60 con 0x7fc7e40cd4a0 2023-09-12T14:55:44.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.684+0000 7fc7dbfff640 1 -- 172.21.15.131:0/3807407298 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc003c00 con 0x7fc7e40cd4a0 2023-09-12T14:55:44.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.684+0000 7fc7e9c31640 1 -- 172.21.15.131:0/3807407298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e41516b0 con 0x7fc7e40cd4a0 2023-09-12T14:55:44.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.684+0000 7fc7e9c31640 1 -- 172.21.15.131:0/3807407298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7e4151230 con 0x7fc7e40cd4a0 2023-09-12T14:55:44.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.684+0000 7fc7dbfff640 1 -- 172.21.15.131:0/3807407298 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fc7dc0b4070 con 0x7fc7e40cd4a0 2023-09-12T14:55:44.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.684+0000 7fc7dbfff640 1 --2- 172.21.15.131:0/3807407298 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc7c4065b20 0x7fc7c4067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:44.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.684+0000 7fc7dbfff640 1 -- 172.21.15.131:0/3807407298 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fc7dc11e040 con 0x7fc7e40cd4a0 2023-09-12T14:55:44.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.684+0000 7fc7e37fe640 1 --2- 172.21.15.131:0/3807407298 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc7c4065b20 0x7fc7c4067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:44.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.684+0000 7fc7e37fe640 1 --2- 172.21.15.131:0/3807407298 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc7c4065b20 0x7fc7c4067fe0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4005c50 tx=0x7fc7d4005be0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:44.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.688+0000 7fc7dbfff640 1 -- 172.21.15.131:0/3807407298 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fc7e4151230 con 0x7fc7e40cd4a0 2023-09-12T14:55:44.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.828+0000 7fc7e9c31640 1 -- 172.21.15.131:0/3807407298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc7e4151230 con 0x7fc7e40cd4a0 2023-09-12T14:55:44.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.828+0000 7fc7dbfff640 1 -- 172.21.15.131:0/3807407298 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc7e4151230 con 0x7fc7e40cd4a0 2023-09-12T14:55:44.831 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-09-12T14:55:44.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.828+0000 7fc7e9c31640 1 -- 172.21.15.131:0/3807407298 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc7c4065b20 msgr2=0x7fc7c4067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:44.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.828+0000 7fc7e9c31640 1 --2- 172.21.15.131:0/3807407298 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc7c4065b20 0x7fc7c4067fe0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4005c50 tx=0x7fc7d4005be0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.828+0000 7fc7e9c31640 1 -- 172.21.15.131:0/3807407298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e40cd4a0 msgr2=0x7fc7e4147920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:44.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.828+0000 7fc7e9c31640 1 --2- 172.21.15.131:0/3807407298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e40cd4a0 0x7fc7e4147920 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc0193f0 tx=0x7fc7dc0a5640 comp rx=0 tx=0).stop 2023-09-12T14:55:44.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.828+0000 7fc7e9c31640 1 -- 172.21.15.131:0/3807407298 shutdown_connections 2023-09-12T14:55:44.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.828+0000 7fc7e9c31640 1 --2- 172.21.15.131:0/3807407298 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc7c4065b20 0x7fc7c4067fe0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.828+0000 7fc7e9c31640 1 --2- 172.21.15.131:0/3807407298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e40cd4a0 0x7fc7e4147920 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.828+0000 7fc7e9c31640 1 --2- 172.21.15.131:0/3807407298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7e40cd0c0 0x7fc7e41594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.828+0000 7fc7e9c31640 1 --2- 172.21.15.131:0/3807407298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7e40aeb90 0x7fc7e40b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.828+0000 7fc7e9c31640 1 -- 172.21.15.131:0/3807407298 >> 172.21.15.131:0/3807407298 conn(0x7fc7e40ceae0 msgr2=0x7fc7e40ad100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:44.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.828+0000 7fc7e9c31640 1 -- 172.21.15.131:0/3807407298 shutdown_connections 2023-09-12T14:55:44.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.828+0000 7fc7e9c31640 1 -- 172.21.15.131:0/3807407298 wait complete. 2023-09-12T14:55:44.848 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T14:55:44.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.972+0000 7fc783a9a640 1 -- 172.21.15.131:0/3501420961 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc77c1574d0 msgr2=0x7fc77c0cacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:44.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.972+0000 7fc783a9a640 1 --2- 172.21.15.131:0/3501420961 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc77c1574d0 0x7fc77c0cacd0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0cd990 tx=0x7fc77801c870 comp rx=0 tx=0).stop 2023-09-12T14:55:44.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.972+0000 7fc783a9a640 1 -- 172.21.15.131:0/3501420961 shutdown_connections 2023-09-12T14:55:44.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.972+0000 7fc783a9a640 1 --2- 172.21.15.131:0/3501420961 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc77c0c98d0 0x7fc77c06aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.972+0000 7fc783a9a640 1 --2- 172.21.15.131:0/3501420961 >> v2:172.21.15.187:3300/0 conn(0x7fc77c0b2600 0x7fc77c0c9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.972+0000 7fc783a9a640 1 --2- 172.21.15.131:0/3501420961 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc77c1574d0 0x7fc77c0cacd0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.972+0000 7fc783a9a640 1 -- 172.21.15.131:0/3501420961 >> 172.21.15.131:0/3501420961 conn(0x7fc77c069840 msgr2=0x7fc77c0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:44.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.972+0000 7fc783a9a640 1 -- 172.21.15.131:0/3501420961 shutdown_connections 2023-09-12T14:55:44.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.972+0000 7fc783a9a640 1 -- 172.21.15.131:0/3501420961 wait complete. 2023-09-12T14:55:44.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.972+0000 7fc783a9a640 1 Processor -- start 2023-09-12T14:55:44.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.972+0000 7fc783a9a640 1 -- start start 2023-09-12T14:55:44.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.976+0000 7fc783a9a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc77c0b2600 0x7fc77c155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:44.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.976+0000 7fc783a9a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc77c0c98d0 0x7fc77c0c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:44.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.976+0000 7fc783a9a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc77c1574d0 0x7fc77c1579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:44.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.976+0000 7fc783a9a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c14b2c0 con 0x7fc77c0b2600 2023-09-12T14:55:44.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.976+0000 7fc783a9a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c153f60 con 0x7fc77c1574d0 2023-09-12T14:55:44.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.976+0000 7fc783a9a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c154240 con 0x7fc77c0c98d0 2023-09-12T14:55:44.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.976+0000 7fc781836640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc77c0b2600 0x7fc77c155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:44.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.976+0000 7fc781836640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc77c0b2600 0x7fc77c155cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48974/0 (socket says 172.21.15.131:48974) 2023-09-12T14:55:44.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.976+0000 7fc781836640 1 -- 172.21.15.131:0/1900000991 learned_addr learned my addr 172.21.15.131:0/1900000991 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:44.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.976+0000 7fc782037640 1 --2- 172.21.15.131:0/1900000991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc77c1574d0 0x7fc77c1579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:44.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.976+0000 7fc781035640 1 --2- 172.21.15.131:0/1900000991 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc77c0c98d0 0x7fc77c0c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:44.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.976+0000 7fc781836640 1 -- 172.21.15.131:0/1900000991 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc77c0c98d0 msgr2=0x7fc77c0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:44.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.976+0000 7fc781836640 1 --2- 172.21.15.131:0/1900000991 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc77c0c98d0 0x7fc77c0c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.976+0000 7fc781836640 1 -- 172.21.15.131:0/1900000991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc77c1574d0 msgr2=0x7fc77c1579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:44.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.976+0000 7fc781836640 1 --2- 172.21.15.131:0/1900000991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc77c1574d0 0x7fc77c1579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:44.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.976+0000 7fc781836640 1 -- 172.21.15.131:0/1900000991 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc778009560 con 0x7fc77c0b2600 2023-09-12T14:55:44.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.976+0000 7fc781836640 1 --2- 172.21.15.131:0/1900000991 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc77c0b2600 0x7fc77c155cd0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc77801af20 tx=0x7fc778005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:44.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.976+0000 7fc7727fc640 1 -- 172.21.15.131:0/1900000991 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc778005230 con 0x7fc77c0b2600 2023-09-12T14:55:44.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.976+0000 7fc7727fc640 1 -- 172.21.15.131:0/1900000991 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc7780054a0 con 0x7fc77c0b2600 2023-09-12T14:55:44.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.976+0000 7fc783a9a640 1 -- 172.21.15.131:0/1900000991 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc77c14b510 con 0x7fc77c0b2600 2023-09-12T14:55:44.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.976+0000 7fc7727fc640 1 -- 172.21.15.131:0/1900000991 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc778004930 con 0x7fc77c0b2600 2023-09-12T14:55:44.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.976+0000 7fc783a9a640 1 -- 172.21.15.131:0/1900000991 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77c14d3f0 con 0x7fc77c0b2600 2023-09-12T14:55:44.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.976+0000 7fc783a9a640 1 -- 172.21.15.131:0/1900000991 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77c14ba30 con 0x7fc77c0b2600 2023-09-12T14:55:44.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.976+0000 7fc7727fc640 1 -- 172.21.15.131:0/1900000991 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fc7780b4040 con 0x7fc77c0b2600 2023-09-12T14:55:44.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.976+0000 7fc7727fc640 1 --2- 172.21.15.131:0/1900000991 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc760065b20 0x7fc760067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:44.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.976+0000 7fc7727fc640 1 -- 172.21.15.131:0/1900000991 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fc77811d500 con 0x7fc77c0b2600 2023-09-12T14:55:44.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.976+0000 7fc781035640 1 --2- 172.21.15.131:0/1900000991 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc760065b20 0x7fc760067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:44.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.980+0000 7fc781035640 1 --2- 172.21.15.131:0/1900000991 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc760065b20 0x7fc760067fe0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc76c005fd0 tx=0x7fc76c005e60 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:44.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:44.980+0000 7fc7727fc640 1 -- 172.21.15.131:0/1900000991 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fc7780eae60 con 0x7fc77c0b2600 2023-09-12T14:55:45.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.124+0000 7fc783a9a640 1 -- 172.21.15.131:0/1900000991 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc77c073580 con 0x7fc77c0b2600 2023-09-12T14:55:45.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.124+0000 7fc7727fc640 1 -- 172.21.15.131:0/1900000991 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc7780ee480 con 0x7fc77c0b2600 2023-09-12T14:55:45.127 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-09-12T14:55:45.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.124+0000 7fc783a9a640 1 -- 172.21.15.131:0/1900000991 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc760065b20 msgr2=0x7fc760067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:45.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.124+0000 7fc783a9a640 1 --2- 172.21.15.131:0/1900000991 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc760065b20 0x7fc760067fe0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc76c005fd0 tx=0x7fc76c005e60 comp rx=0 tx=0).stop 2023-09-12T14:55:45.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.124+0000 7fc783a9a640 1 -- 172.21.15.131:0/1900000991 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc77c0b2600 msgr2=0x7fc77c155cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:45.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.124+0000 7fc783a9a640 1 --2- 172.21.15.131:0/1900000991 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc77c0b2600 0x7fc77c155cd0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc77801af20 tx=0x7fc778005e90 comp rx=0 tx=0).stop 2023-09-12T14:55:45.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.124+0000 7fc783a9a640 1 -- 172.21.15.131:0/1900000991 shutdown_connections 2023-09-12T14:55:45.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.124+0000 7fc783a9a640 1 --2- 172.21.15.131:0/1900000991 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc760065b20 0x7fc760067fe0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:45.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.124+0000 7fc783a9a640 1 --2- 172.21.15.131:0/1900000991 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc77c0c98d0 0x7fc77c0c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:45.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.124+0000 7fc783a9a640 1 --2- 172.21.15.131:0/1900000991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc77c1574d0 0x7fc77c1579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:45.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.124+0000 7fc783a9a640 1 --2- 172.21.15.131:0/1900000991 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc77c0b2600 0x7fc77c155cd0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:45.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.124+0000 7fc783a9a640 1 -- 172.21.15.131:0/1900000991 >> 172.21.15.131:0/1900000991 conn(0x7fc77c069840 msgr2=0x7fc77c0ad720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:45.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.124+0000 7fc783a9a640 1 -- 172.21.15.131:0/1900000991 shutdown_connections 2023-09-12T14:55:45.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.124+0000 7fc783a9a640 1 -- 172.21.15.131:0/1900000991 wait complete. 2023-09-12T14:55:45.144 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T14:55:45.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5703cd5640 1 -- 172.21.15.131:0/757892765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56fc06c3b0 msgr2=0x7f56fc06fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:45.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5703cd5640 1 --2- 172.21.15.131:0/757892765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56fc06c3b0 0x7f56fc06fdd0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f56fc070930 tx=0x7f56f401c870 comp rx=0 tx=0).stop 2023-09-12T14:55:45.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5703cd5640 1 -- 172.21.15.131:0/757892765 shutdown_connections 2023-09-12T14:55:45.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5703cd5640 1 --2- 172.21.15.131:0/757892765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56fc06c3b0 0x7f56fc06fdd0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:45.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5703cd5640 1 --2- 172.21.15.131:0/757892765 >> v2:172.21.15.187:3300/0 conn(0x7f56fc0b2100 0x7f56fc06be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:45.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5703cd5640 1 --2- 172.21.15.131:0/757892765 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56fc1556e0 0x7f56fc158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:45.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5703cd5640 1 -- 172.21.15.131:0/757892765 >> 172.21.15.131:0/757892765 conn(0x7f56fc06a950 msgr2=0x7f56fc0af5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:45.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5703cd5640 1 -- 172.21.15.131:0/757892765 shutdown_connections 2023-09-12T14:55:45.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5703cd5640 1 -- 172.21.15.131:0/757892765 wait complete. 2023-09-12T14:55:45.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5703cd5640 1 Processor -- start 2023-09-12T14:55:45.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5703cd5640 1 -- start start 2023-09-12T14:55:45.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5703cd5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56fc06c3b0 0x7f56fc14c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:45.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5703cd5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56fc0b2100 0x7f56fc161540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:45.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5703cd5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56fc1556e0 0x7f56fc162880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:45.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5703cd5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc1f3df0 con 0x7f56fc06c3b0 2023-09-12T14:55:45.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5703cd5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc161a80 con 0x7f56fc1556e0 2023-09-12T14:55:45.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5703cd5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc1630b0 con 0x7f56fc0b2100 2023-09-12T14:55:45.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5701a71640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56fc06c3b0 0x7f56fc14c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:45.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5701a71640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56fc06c3b0 0x7f56fc14c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48982/0 (socket says 172.21.15.131:48982) 2023-09-12T14:55:45.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5701a71640 1 -- 172.21.15.131:0/2858421815 learned_addr learned my addr 172.21.15.131:0/2858421815 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:45.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5702272640 1 --2- 172.21.15.131:0/2858421815 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56fc1556e0 0x7f56fc162880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:45.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5701a71640 1 -- 172.21.15.131:0/2858421815 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56fc0b2100 msgr2=0x7f56fc161540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:45.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5701a71640 1 --2- 172.21.15.131:0/2858421815 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56fc0b2100 0x7f56fc161540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:45.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5701a71640 1 -- 172.21.15.131:0/2858421815 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56fc1556e0 msgr2=0x7f56fc162880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:45.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5701a71640 1 --2- 172.21.15.131:0/2858421815 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56fc1556e0 0x7f56fc162880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:45.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5701a71640 1 -- 172.21.15.131:0/2858421815 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f4009560 con 0x7f56fc06c3b0 2023-09-12T14:55:45.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5701a71640 1 --2- 172.21.15.131:0/2858421815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56fc06c3b0 0x7f56fc14c7f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f56f800ba50 tx=0x7f56f800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:45.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f56f27fc640 1 -- 172.21.15.131:0/2858421815 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f800cbe0 con 0x7f56fc06c3b0 2023-09-12T14:55:45.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f56f27fc640 1 -- 172.21.15.131:0/2858421815 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f56f800ce50 con 0x7f56fc06c3b0 2023-09-12T14:55:45.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5703cd5640 1 -- 172.21.15.131:0/2858421815 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56fc1f4030 con 0x7f56fc06c3b0 2023-09-12T14:55:45.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f56f27fc640 1 -- 172.21.15.131:0/2858421815 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f8014680 con 0x7f56fc06c3b0 2023-09-12T14:55:45.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.264+0000 7f5703cd5640 1 -- 172.21.15.131:0/2858421815 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56fc1f45f0 con 0x7f56fc06c3b0 2023-09-12T14:55:45.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.268+0000 7f5703cd5640 1 -- 172.21.15.131:0/2858421815 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56fc073580 con 0x7f56fc06c3b0 2023-09-12T14:55:45.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.268+0000 7f56f27fc640 1 -- 172.21.15.131:0/2858421815 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f56f8007370 con 0x7f56fc06c3b0 2023-09-12T14:55:45.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.268+0000 7f56f27fc640 1 --2- 172.21.15.131:0/2858421815 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f56e0065b20 0x7f56e0067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:45.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.268+0000 7f56f27fc640 1 -- 172.21.15.131:0/2858421815 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f56f80835a0 con 0x7f56fc06c3b0 2023-09-12T14:55:45.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.268+0000 7f5701270640 1 --2- 172.21.15.131:0/2858421815 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f56e0065b20 0x7f56e0067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:45.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.268+0000 7f5701270640 1 --2- 172.21.15.131:0/2858421815 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f56e0065b20 0x7f56e0067fe0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f56fc0b0870 tx=0x7f56ec005c00 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:45.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.272+0000 7f56f27fc640 1 -- 172.21.15.131:0/2858421815 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f56f8050e70 con 0x7f56fc06c3b0 2023-09-12T14:55:45.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.460+0000 7f5703cd5640 1 -- 172.21.15.131:0/2858421815 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56fc162dc0 con 0x7f56fc06c3b0 2023-09-12T14:55:45.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.460+0000 7f56f27fc640 1 -- 172.21.15.131:0/2858421815 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f56f8054490 con 0x7f56fc06c3b0 2023-09-12T14:55:45.463 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:45.464 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4344,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/491327908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":491327908},{"type":"v1","addr":"172.21.15.131:6837","nonce":491327908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3470212105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3470212105},{"type":"v1","addr":"172.21.15.187:6833","nonce":3470212105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/91478708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":91478708},{"type":"v1","addr":"172.21.15.131:6836","nonce":91478708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-12T14:54:58.030444+0000","modified":"2023-09-12T14:55:00.236338+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4396":{"gid":4396,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/290727181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":290727181},{"type":"v1","addr":"172.21.15.187:6835","nonce":290727181}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-12T14:55:45.464 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 6 2023-09-12T14:55:45.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.460+0000 7f5703cd5640 1 -- 172.21.15.131:0/2858421815 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f56e0065b20 msgr2=0x7f56e0067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:45.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.460+0000 7f5703cd5640 1 --2- 172.21.15.131:0/2858421815 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f56e0065b20 0x7f56e0067fe0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f56fc0b0870 tx=0x7f56ec005c00 comp rx=0 tx=0).stop 2023-09-12T14:55:45.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.460+0000 7f5703cd5640 1 -- 172.21.15.131:0/2858421815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56fc06c3b0 msgr2=0x7f56fc14c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:45.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.460+0000 7f5703cd5640 1 --2- 172.21.15.131:0/2858421815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56fc06c3b0 0x7f56fc14c7f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f56f800ba50 tx=0x7f56f800bf10 comp rx=0 tx=0).stop 2023-09-12T14:55:45.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.460+0000 7f5703cd5640 1 -- 172.21.15.131:0/2858421815 shutdown_connections 2023-09-12T14:55:45.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.460+0000 7f5703cd5640 1 --2- 172.21.15.131:0/2858421815 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f56e0065b20 0x7f56e0067fe0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:45.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.460+0000 7f5703cd5640 1 --2- 172.21.15.131:0/2858421815 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56fc0b2100 0x7f56fc161540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:45.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.460+0000 7f5703cd5640 1 --2- 172.21.15.131:0/2858421815 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56fc1556e0 0x7f56fc162880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:45.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.460+0000 7f5703cd5640 1 --2- 172.21.15.131:0/2858421815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56fc06c3b0 0x7f56fc14c7f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:45.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.460+0000 7f5703cd5640 1 -- 172.21.15.131:0/2858421815 >> 172.21.15.131:0/2858421815 conn(0x7f56fc06a950 msgr2=0x7f56fc161ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:45.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.460+0000 7f5703cd5640 1 -- 172.21.15.131:0/2858421815 shutdown_connections 2023-09-12T14:55:45.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.460+0000 7f5703cd5640 1 -- 172.21.15.131:0/2858421815 wait complete. 2023-09-12T14:55:45.482 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:55:45.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad9b339640 1 -- 172.21.15.131:0/276857200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad9406f390 msgr2=0x7fad9406f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:45.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad9b339640 1 --2- 172.21.15.131:0/276857200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad9406f390 0x7fad9406f820 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fad9406eea0 tx=0x7fad8c01cd30 comp rx=0 tx=0).stop 2023-09-12T14:55:45.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad9b339640 1 -- 172.21.15.131:0/276857200 shutdown_connections 2023-09-12T14:55:45.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad9b339640 1 --2- 172.21.15.131:0/276857200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad9406f390 0x7fad9406f820 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:45.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad9b339640 1 --2- 172.21.15.131:0/276857200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad94158180 0x7fad940cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:45.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad9b339640 1 --2- 172.21.15.131:0/276857200 >> v2:172.21.15.187:3300/0 conn(0x7fad940ab730 0x7fad940abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:45.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad9b339640 1 -- 172.21.15.131:0/276857200 >> 172.21.15.131:0/276857200 conn(0x7fad94069840 msgr2=0x7fad940cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:45.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad9b339640 1 -- 172.21.15.131:0/276857200 shutdown_connections 2023-09-12T14:55:45.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad9b339640 1 -- 172.21.15.131:0/276857200 wait complete. 2023-09-12T14:55:45.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad9b339640 1 Processor -- start 2023-09-12T14:55:45.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad9b339640 1 -- start start 2023-09-12T14:55:45.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad9b339640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad9406f390 0x7fad940b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:45.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad9b339640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad940ab730 0x7fad940b2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:45.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad9b339640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad94158180 0x7fad9414e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:45.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad9b339640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad940b1740 con 0x7fad94158180 2023-09-12T14:55:45.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad9b339640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad940a9b80 con 0x7fad940ab730 2023-09-12T14:55:45.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad9b339640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad940b25a0 con 0x7fad9406f390 2023-09-12T14:55:45.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad998d6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad94158180 0x7fad9414e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:45.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad998d6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad94158180 0x7fad9414e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48994/0 (socket says 172.21.15.131:48994) 2023-09-12T14:55:45.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad998d6640 1 -- 172.21.15.131:0/2736716280 learned_addr learned my addr 172.21.15.131:0/2736716280 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:45.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad998d6640 1 -- 172.21.15.131:0/2736716280 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad9406f390 msgr2=0x7fad940b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:45.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad998d6640 1 --2- 172.21.15.131:0/2736716280 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad9406f390 0x7fad940b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:45.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad998d6640 1 -- 172.21.15.131:0/2736716280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad940ab730 msgr2=0x7fad940b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:45.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad988d4640 1 --2- 172.21.15.131:0/2736716280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad940ab730 0x7fad940b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:45.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad998d6640 1 --2- 172.21.15.131:0/2736716280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad940ab730 0x7fad940b2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:45.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad998d6640 1 -- 172.21.15.131:0/2736716280 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c0098e0 con 0x7fad94158180 2023-09-12T14:55:45.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad998d6640 1 --2- 172.21.15.131:0/2736716280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad94158180 0x7fad9414e2a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fad8c009bd0 tx=0x7fad8c004b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:45.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad89ffb640 1 -- 172.21.15.131:0/2736716280 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c003980 con 0x7fad94158180 2023-09-12T14:55:45.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad89ffb640 1 -- 172.21.15.131:0/2736716280 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fad8c003bd0 con 0x7fad94158180 2023-09-12T14:55:45.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad9b339640 1 -- 172.21.15.131:0/2736716280 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad940a9570 con 0x7fad94158180 2023-09-12T14:55:45.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.592+0000 7fad89ffb640 1 -- 172.21.15.131:0/2736716280 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c0ad6a0 con 0x7fad94158180 2023-09-12T14:55:45.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.596+0000 7fad9b339640 1 -- 172.21.15.131:0/2736716280 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad940b1460 con 0x7fad94158180 2023-09-12T14:55:45.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.596+0000 7fad9b339640 1 -- 172.21.15.131:0/2736716280 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad9406f980 con 0x7fad94158180 2023-09-12T14:55:45.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.596+0000 7fad89ffb640 1 -- 172.21.15.131:0/2736716280 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fad8c0b4040 con 0x7fad94158180 2023-09-12T14:55:45.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.596+0000 7fad89ffb640 1 --2- 172.21.15.131:0/2736716280 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fad78065b20 0x7fad78067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:45.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.596+0000 7fad89ffb640 1 -- 172.21.15.131:0/2736716280 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fad8c11dbc0 con 0x7fad94158180 2023-09-12T14:55:45.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.596+0000 7fad990d5640 1 --2- 172.21.15.131:0/2736716280 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fad78065b20 0x7fad78067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:45.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.596+0000 7fad990d5640 1 --2- 172.21.15.131:0/2736716280 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fad78065b20 0x7fad78067fe0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fad90009ae0 tx=0x7fad900095a0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:45.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.600+0000 7fad89ffb640 1 -- 172.21.15.131:0/2736716280 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fad8c0eb520 con 0x7fad94158180 2023-09-12T14:55:45.750 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.748+0000 7fad9b339640 1 -- 172.21.15.131:0/2736716280 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad94152550 con 0x7fad94158180 2023-09-12T14:55:45.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.748+0000 7fad89ffb640 1 -- 172.21.15.131:0/2736716280 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16791 (secure 0 0 0) 0x7fad8c0b2090 con 0x7fad94158180 2023-09-12T14:55:45.753 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:45.754 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":22,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:55:39.097847+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:54:55.689983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T14:54:57.029447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:55:45.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.752+0000 7fad9b339640 1 -- 172.21.15.131:0/2736716280 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fad78065b20 msgr2=0x7fad78067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:45.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.752+0000 7fad9b339640 1 --2- 172.21.15.131:0/2736716280 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fad78065b20 0x7fad78067fe0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fad90009ae0 tx=0x7fad900095a0 comp rx=0 tx=0).stop 2023-09-12T14:55:45.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.752+0000 7fad9b339640 1 -- 172.21.15.131:0/2736716280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad94158180 msgr2=0x7fad9414e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:45.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.752+0000 7fad9b339640 1 --2- 172.21.15.131:0/2736716280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad94158180 0x7fad9414e2a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fad8c009bd0 tx=0x7fad8c004b90 comp rx=0 tx=0).stop 2023-09-12T14:55:45.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.752+0000 7fad9b339640 1 -- 172.21.15.131:0/2736716280 shutdown_connections 2023-09-12T14:55:45.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.752+0000 7fad9b339640 1 --2- 172.21.15.131:0/2736716280 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fad78065b20 0x7fad78067fe0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:45.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.752+0000 7fad9b339640 1 --2- 172.21.15.131:0/2736716280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad94158180 0x7fad9414e2a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:45.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.752+0000 7fad9b339640 1 --2- 172.21.15.131:0/2736716280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad940ab730 0x7fad940b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:45.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.752+0000 7fad9b339640 1 --2- 172.21.15.131:0/2736716280 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad9406f390 0x7fad940b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:45.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.752+0000 7fad9b339640 1 -- 172.21.15.131:0/2736716280 >> 172.21.15.131:0/2736716280 conn(0x7fad94069840 msgr2=0x7fad9406bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:45.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.752+0000 7fad9b339640 1 -- 172.21.15.131:0/2736716280 shutdown_connections 2023-09-12T14:55:45.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.752+0000 7fad9b339640 1 -- 172.21.15.131:0/2736716280 wait complete. 2023-09-12T14:55:45.771 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-12T14:55:45.771 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T14:55:45.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.888+0000 7fc672deb640 1 -- 172.21.15.131:0/2010524284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc66c06b050 msgr2=0x7fc66c06b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:45.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.888+0000 7fc672deb640 1 --2- 172.21.15.131:0/2010524284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc66c06b050 0x7fc66c06b4e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009c80 tx=0x7fc65c01cd50 comp rx=0 tx=0).stop 2023-09-12T14:55:45.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.888+0000 7fc672deb640 1 -- 172.21.15.131:0/2010524284 shutdown_connections 2023-09-12T14:55:45.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.888+0000 7fc672deb640 1 --2- 172.21.15.131:0/2010524284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc66c06b050 0x7fc66c06b4e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:45.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.888+0000 7fc672deb640 1 --2- 172.21.15.131:0/2010524284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc66c0ab730 0x7fc66c0abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:45.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.888+0000 7fc672deb640 1 -- 172.21.15.131:0/2010524284 >> 172.21.15.131:0/2010524284 conn(0x7fc66c069840 msgr2=0x7fc66c0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:45.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.888+0000 7fc672deb640 1 -- 172.21.15.131:0/2010524284 shutdown_connections 2023-09-12T14:55:45.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.888+0000 7fc672deb640 1 -- 172.21.15.131:0/2010524284 wait complete. 2023-09-12T14:55:45.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.888+0000 7fc672deb640 1 Processor -- start 2023-09-12T14:55:45.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.888+0000 7fc672deb640 1 -- start start 2023-09-12T14:55:45.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.888+0000 7fc672deb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc66c06b050 0x7fc66c148660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:45.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.888+0000 7fc672deb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc66c0ab730 0x7fc66c14dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:45.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.888+0000 7fc672deb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc66c158180 0x7fc66c13c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:45.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.888+0000 7fc672deb640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c13c6e0 con 0x7fc66c0ab730 2023-09-12T14:55:45.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.888+0000 7fc672deb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c14c4c0 con 0x7fc66c158180 2023-09-12T14:55:45.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.888+0000 7fc672deb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c14b8b0 con 0x7fc66c06b050 2023-09-12T14:55:45.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.888+0000 7fc66bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc66c0ab730 0x7fc66c14dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:45.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.888+0000 7fc66bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc66c0ab730 0x7fc66c14dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49026/0 (socket says 172.21.15.131:49026) 2023-09-12T14:55:45.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.888+0000 7fc66bfff640 1 -- 172.21.15.131:0/1271977302 learned_addr learned my addr 172.21.15.131:0/1271977302 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:45.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.888+0000 7fc670b87640 1 --2- 172.21.15.131:0/1271977302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc66c06b050 0x7fc66c148660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:45.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.888+0000 7fc671388640 1 --2- 172.21.15.131:0/1271977302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc66c158180 0x7fc66c13c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:45.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.888+0000 7fc66bfff640 1 -- 172.21.15.131:0/1271977302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc66c06b050 msgr2=0x7fc66c148660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:45.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.888+0000 7fc66bfff640 1 --2- 172.21.15.131:0/1271977302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc66c06b050 0x7fc66c148660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:45.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.888+0000 7fc66bfff640 1 -- 172.21.15.131:0/1271977302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc66c158180 msgr2=0x7fc66c13c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:45.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.888+0000 7fc66bfff640 1 --2- 172.21.15.131:0/1271977302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc66c158180 0x7fc66c13c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:45.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.888+0000 7fc66bfff640 1 -- 172.21.15.131:0/1271977302 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc65c0098e0 con 0x7fc66c0ab730 2023-09-12T14:55:45.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.888+0000 7fc670b87640 1 --2- 172.21.15.131:0/1271977302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc66c06b050 0x7fc66c148660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:45.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.892+0000 7fc66bfff640 1 --2- 172.21.15.131:0/1271977302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc66c0ab730 0x7fc66c14dca0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc660001ec0 tx=0x7fc66000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:45.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.892+0000 7fc6697fa640 1 -- 172.21.15.131:0/1271977302 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6600054e0 con 0x7fc66c0ab730 2023-09-12T14:55:45.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.892+0000 7fc6697fa640 1 -- 172.21.15.131:0/1271977302 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc660003600 con 0x7fc66c0ab730 2023-09-12T14:55:45.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.892+0000 7fc672deb640 1 -- 172.21.15.131:0/1271977302 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc66c14bb90 con 0x7fc66c0ab730 2023-09-12T14:55:45.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.892+0000 7fc6697fa640 1 -- 172.21.15.131:0/1271977302 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc660003a10 con 0x7fc66c0ab730 2023-09-12T14:55:45.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.892+0000 7fc672deb640 1 -- 172.21.15.131:0/1271977302 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc66c14c150 con 0x7fc66c0ab730 2023-09-12T14:55:45.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.892+0000 7fc672deb640 1 -- 172.21.15.131:0/1271977302 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc66c073580 con 0x7fc66c0ab730 2023-09-12T14:55:45.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.892+0000 7fc6697fa640 1 -- 172.21.15.131:0/1271977302 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fc660003b70 con 0x7fc66c0ab730 2023-09-12T14:55:45.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.892+0000 7fc6697fa640 1 --2- 172.21.15.131:0/1271977302 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc658065b50 0x7fc658068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:45.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.892+0000 7fc6697fa640 1 -- 172.21.15.131:0/1271977302 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fc660082a80 con 0x7fc66c0ab730 2023-09-12T14:55:45.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.892+0000 7fc670b87640 1 --2- 172.21.15.131:0/1271977302 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc658065b50 0x7fc658068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:45.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.892+0000 7fc670b87640 1 --2- 172.21.15.131:0/1271977302 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc658065b50 0x7fc658068010 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc664005fd0 tx=0x7fc664005da0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:45.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:45.896+0000 7fc6697fa640 1 -- 172.21.15.131:0/1271977302 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fc660050350 con 0x7fc66c0ab730 2023-09-12T14:55:46.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.068+0000 7fc672deb640 1 -- 172.21.15.131:0/1271977302 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc66c0ca020 con 0x7fc66c0ab730 2023-09-12T14:55:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.072+0000 7fc6697fa640 1 -- 172.21.15.131:0/1271977302 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4907 (secure 0 0 0) 0x7fc660017090 con 0x7fc66c0ab730 2023-09-12T14:55:46.075 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:46.075 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4344,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/491327908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":491327908},{"type":"v1","addr":"172.21.15.131:6837","nonce":491327908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3470212105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3470212105},{"type":"v1","addr":"172.21.15.187:6833","nonce":3470212105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/91478708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":91478708},{"type":"v1","addr":"172.21.15.131:6836","nonce":91478708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-12T14:54:58.030444+0000","modified":"2023-09-12T14:55:00.236338+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4396":{"gid":4396,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/290727181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":290727181},{"type":"v1","addr":"172.21.15.187:6835","nonce":290727181}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-12T14:55:46.075 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 6 2023-09-12T14:55:46.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.076+0000 7fc672deb640 1 -- 172.21.15.131:0/1271977302 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc658065b50 msgr2=0x7fc658068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:46.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.076+0000 7fc672deb640 1 --2- 172.21.15.131:0/1271977302 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc658065b50 0x7fc658068010 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc664005fd0 tx=0x7fc664005da0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.076+0000 7fc672deb640 1 -- 172.21.15.131:0/1271977302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc66c0ab730 msgr2=0x7fc66c14dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:46.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.076+0000 7fc672deb640 1 --2- 172.21.15.131:0/1271977302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc66c0ab730 0x7fc66c14dca0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc660001ec0 tx=0x7fc66000c8e0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.076+0000 7fc672deb640 1 -- 172.21.15.131:0/1271977302 shutdown_connections 2023-09-12T14:55:46.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.076+0000 7fc672deb640 1 --2- 172.21.15.131:0/1271977302 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc658065b50 0x7fc658068010 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.076+0000 7fc672deb640 1 --2- 172.21.15.131:0/1271977302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc66c158180 0x7fc66c13c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.076+0000 7fc672deb640 1 --2- 172.21.15.131:0/1271977302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc66c0ab730 0x7fc66c14dca0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.076+0000 7fc672deb640 1 --2- 172.21.15.131:0/1271977302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc66c06b050 0x7fc66c148660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.076+0000 7fc672deb640 1 -- 172.21.15.131:0/1271977302 >> 172.21.15.131:0/1271977302 conn(0x7fc66c069840 msgr2=0x7fc66c06f920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:46.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.076+0000 7fc672deb640 1 -- 172.21.15.131:0/1271977302 shutdown_connections 2023-09-12T14:55:46.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.076+0000 7fc672deb640 1 -- 172.21.15.131:0/1271977302 wait complete. 2023-09-12T14:55:46.093 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T14:55:46.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4f4c10640 1 -- 172.21.15.131:0/1769992349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4f00c9a20 msgr2=0x7fc4f015ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:46.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4f4c10640 1 --2- 172.21.15.131:0/1769992349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4f00c9a20 0x7fc4f015ca30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0071110 tx=0x7fc4e801cad0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4f4c10640 1 -- 172.21.15.131:0/1769992349 shutdown_connections 2023-09-12T14:55:46.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4f4c10640 1 --2- 172.21.15.131:0/1769992349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4f00c9a20 0x7fc4f015ca30 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4f4c10640 1 --2- 172.21.15.131:0/1769992349 >> v2:172.21.15.187:3300/0 conn(0x7fc4f0158d70 0x7fc4f006b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4f4c10640 1 -- 172.21.15.131:0/1769992349 >> 172.21.15.131:0/1769992349 conn(0x7fc4f0069840 msgr2=0x7fc4f00cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:46.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4f4c10640 1 -- 172.21.15.131:0/1769992349 shutdown_connections 2023-09-12T14:55:46.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4f4c10640 1 -- 172.21.15.131:0/1769992349 wait complete. 2023-09-12T14:55:46.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4f4c10640 1 Processor -- start 2023-09-12T14:55:46.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4f4c10640 1 -- start start 2023-09-12T14:55:46.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4f4c10640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f00abae0 0x7fc4f01424a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:46.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4f4c10640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4f00c9a20 0x7fc4f01518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:46.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4f4c10640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4f0158d70 0x7fc4f00aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:46.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4f4c10640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f01525b0 con 0x7fc4f00c9a20 2023-09-12T14:55:46.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4f4c10640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f0153290 con 0x7fc4f0158d70 2023-09-12T14:55:46.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4f4c10640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f0158150 con 0x7fc4f00abae0 2023-09-12T14:55:46.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4edd9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4f00c9a20 0x7fc4f01518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:46.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4edd9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4f00c9a20 0x7fc4f01518f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49048/0 (socket says 172.21.15.131:49048) 2023-09-12T14:55:46.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4edd9b640 1 -- 172.21.15.131:0/3420588152 learned_addr learned my addr 172.21.15.131:0/3420588152 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:46.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4edd9b640 1 -- 172.21.15.131:0/3420588152 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f00abae0 msgr2=0x7fc4f01424a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:46.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4edd9b640 1 --2- 172.21.15.131:0/3420588152 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f00abae0 0x7fc4f01424a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4edd9b640 1 -- 172.21.15.131:0/3420588152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4f0158d70 msgr2=0x7fc4f00aa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:46.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4edd9b640 1 --2- 172.21.15.131:0/3420588152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4f0158d70 0x7fc4f00aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4edd9b640 1 -- 172.21.15.131:0/3420588152 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e8009820 con 0x7fc4f00c9a20 2023-09-12T14:55:46.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4edd9b640 1 --2- 172.21.15.131:0/3420588152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4f00c9a20 0x7fc4f01518f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400c4b0 tx=0x7fc4e400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:46.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4deffd640 1 -- 172.21.15.131:0/3420588152 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4005480 con 0x7fc4f00c9a20 2023-09-12T14:55:46.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4deffd640 1 -- 172.21.15.131:0/3420588152 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc4e4003600 con 0x7fc4f00c9a20 2023-09-12T14:55:46.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4f4c10640 1 -- 172.21.15.131:0/3420588152 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f0151ff0 con 0x7fc4f00c9a20 2023-09-12T14:55:46.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4deffd640 1 -- 172.21.15.131:0/3420588152 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4003a10 con 0x7fc4f00c9a20 2023-09-12T14:55:46.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4f4c10640 1 -- 172.21.15.131:0/3420588152 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f0152f20 con 0x7fc4f00c9a20 2023-09-12T14:55:46.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.212+0000 7fc4f4c10640 1 -- 172.21.15.131:0/3420588152 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f0073580 con 0x7fc4f00c9a20 2023-09-12T14:55:46.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.216+0000 7fc4deffd640 1 -- 172.21.15.131:0/3420588152 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fc4e4003b70 con 0x7fc4f00c9a20 2023-09-12T14:55:46.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.216+0000 7fc4deffd640 1 --2- 172.21.15.131:0/3420588152 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc4d0065b20 0x7fc4d0067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:46.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.216+0000 7fc4deffd640 1 -- 172.21.15.131:0/3420588152 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fc4e4083090 con 0x7fc4f00c9a20 2023-09-12T14:55:46.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.216+0000 7fc4ee59c640 1 --2- 172.21.15.131:0/3420588152 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc4d0065b20 0x7fc4d0067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:46.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.216+0000 7fc4ee59c640 1 --2- 172.21.15.131:0/3420588152 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc4d0065b20 0x7fc4d0067fe0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000a910 tx=0x7fc4e0005be0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:46.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.220+0000 7fc4deffd640 1 -- 172.21.15.131:0/3420588152 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fc4e4050960 con 0x7fc4f00c9a20 2023-09-12T14:55:46.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.392+0000 7fc4f4c10640 1 -- 172.21.15.131:0/3420588152 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4f00cd4d0 con 0x7fc4f00c9a20 2023-09-12T14:55:46.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.392+0000 7fc4deffd640 1 -- 172.21.15.131:0/3420588152 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4907 (secure 0 0 0) 0x7fc4e4053f80 con 0x7fc4f00c9a20 2023-09-12T14:55:46.396 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:46.396 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4344,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/491327908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":491327908},{"type":"v1","addr":"172.21.15.131:6837","nonce":491327908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3470212105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3470212105},{"type":"v1","addr":"172.21.15.187:6833","nonce":3470212105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/91478708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":91478708},{"type":"v1","addr":"172.21.15.131:6836","nonce":91478708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-12T14:54:58.030444+0000","modified":"2023-09-12T14:55:00.236338+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4396":{"gid":4396,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/290727181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":290727181},{"type":"v1","addr":"172.21.15.187:6835","nonce":290727181}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-12T14:55:46.396 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 6 2023-09-12T14:55:46.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.392+0000 7fc4f4c10640 1 -- 172.21.15.131:0/3420588152 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc4d0065b20 msgr2=0x7fc4d0067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:46.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.392+0000 7fc4f4c10640 1 --2- 172.21.15.131:0/3420588152 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc4d0065b20 0x7fc4d0067fe0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000a910 tx=0x7fc4e0005be0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.392+0000 7fc4f4c10640 1 -- 172.21.15.131:0/3420588152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4f00c9a20 msgr2=0x7fc4f01518f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:46.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.392+0000 7fc4f4c10640 1 --2- 172.21.15.131:0/3420588152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4f00c9a20 0x7fc4f01518f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400c4b0 tx=0x7fc4e400c970 comp rx=0 tx=0).stop 2023-09-12T14:55:46.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.392+0000 7fc4f4c10640 1 -- 172.21.15.131:0/3420588152 shutdown_connections 2023-09-12T14:55:46.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.392+0000 7fc4f4c10640 1 --2- 172.21.15.131:0/3420588152 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc4d0065b20 0x7fc4d0067fe0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.392+0000 7fc4f4c10640 1 --2- 172.21.15.131:0/3420588152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4f0158d70 0x7fc4f00aa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.392+0000 7fc4f4c10640 1 --2- 172.21.15.131:0/3420588152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4f00c9a20 0x7fc4f01518f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.392+0000 7fc4f4c10640 1 --2- 172.21.15.131:0/3420588152 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f00abae0 0x7fc4f01424a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.392+0000 7fc4f4c10640 1 -- 172.21.15.131:0/3420588152 >> 172.21.15.131:0/3420588152 conn(0x7fc4f0069840 msgr2=0x7fc4f00cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:46.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.392+0000 7fc4f4c10640 1 -- 172.21.15.131:0/3420588152 shutdown_connections 2023-09-12T14:55:46.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.392+0000 7fc4f4c10640 1 -- 172.21.15.131:0/3420588152 wait complete. 2023-09-12T14:55:46.412 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:55:46.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.532+0000 7f7423d35640 1 -- 172.21.15.131:0/3635211143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f741c1553b0 msgr2=0x7f741c0afa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:46.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.532+0000 7f7423d35640 1 --2- 172.21.15.131:0/3635211143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f741c1553b0 0x7f741c0afa30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7418009900 tx=0x7f741801c880 comp rx=0 tx=0).stop 2023-09-12T14:55:46.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.532+0000 7f7423d35640 1 -- 172.21.15.131:0/3635211143 shutdown_connections 2023-09-12T14:55:46.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.532+0000 7f7423d35640 1 --2- 172.21.15.131:0/3635211143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f741c0cc620 0x7f741c06be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.532+0000 7f7423d35640 1 --2- 172.21.15.131:0/3635211143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f741c1553b0 0x7f741c0afa30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.532+0000 7f7423d35640 1 -- 172.21.15.131:0/3635211143 >> 172.21.15.131:0/3635211143 conn(0x7f741c06a520 msgr2=0x7f741c06a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:46.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.532+0000 7f7423d35640 1 -- 172.21.15.131:0/3635211143 shutdown_connections 2023-09-12T14:55:46.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.532+0000 7f7423d35640 1 -- 172.21.15.131:0/3635211143 wait complete. 2023-09-12T14:55:46.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.532+0000 7f7423d35640 1 Processor -- start 2023-09-12T14:55:46.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.532+0000 7f7423d35640 1 -- start start 2023-09-12T14:55:46.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.532+0000 7f7423d35640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f741c0aff70 0x7f741c145fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:46.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.532+0000 7f7423d35640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f741c0cc620 0x7f741c156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:46.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.532+0000 7f7423d35640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f741c1553b0 0x7f741c157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:46.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.532+0000 7f7423d35640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741c14e570 con 0x7f741c0aff70 2023-09-12T14:55:46.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.532+0000 7f7423d35640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741c148890 con 0x7f741c1553b0 2023-09-12T14:55:46.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.532+0000 7f7423d35640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f741c149d00 con 0x7f741c0cc620 2023-09-12T14:55:46.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.532+0000 7f7421ad1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f741c0aff70 0x7f741c145fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:46.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.532+0000 7f7421ad1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f741c0aff70 0x7f741c145fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49074/0 (socket says 172.21.15.131:49074) 2023-09-12T14:55:46.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.532+0000 7f7421ad1640 1 -- 172.21.15.131:0/4098789145 learned_addr learned my addr 172.21.15.131:0/4098789145 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:46.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.536+0000 7f74222d2640 1 --2- 172.21.15.131:0/4098789145 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f741c1553b0 0x7f741c157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:46.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.536+0000 7f7421ad1640 1 -- 172.21.15.131:0/4098789145 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f741c0cc620 msgr2=0x7f741c156cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:46.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.536+0000 7f74212d0640 1 --2- 172.21.15.131:0/4098789145 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f741c0cc620 0x7f741c156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:46.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.536+0000 7f7421ad1640 1 --2- 172.21.15.131:0/4098789145 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f741c0cc620 0x7f741c156cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.536+0000 7f7421ad1640 1 -- 172.21.15.131:0/4098789145 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f741c1553b0 msgr2=0x7f741c157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:46.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.536+0000 7f7421ad1640 1 --2- 172.21.15.131:0/4098789145 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f741c1553b0 0x7f741c157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.536+0000 7f7421ad1640 1 -- 172.21.15.131:0/4098789145 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7418009560 con 0x7f741c0aff70 2023-09-12T14:55:46.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.536+0000 7f74222d2640 1 --2- 172.21.15.131:0/4098789145 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f741c1553b0 0x7f741c157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:46.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.536+0000 7f7421ad1640 1 --2- 172.21.15.131:0/4098789145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f741c0aff70 0x7f741c145fa0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f741801af30 tx=0x7f74180a6780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:46.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.536+0000 7f74127fc640 1 -- 172.21.15.131:0/4098789145 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7418004340 con 0x7f741c0aff70 2023-09-12T14:55:46.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.536+0000 7f74127fc640 1 -- 172.21.15.131:0/4098789145 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7418003600 con 0x7f741c0aff70 2023-09-12T14:55:46.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.536+0000 7f7423d35640 1 -- 172.21.15.131:0/4098789145 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f741c06d040 con 0x7f741c0aff70 2023-09-12T14:55:46.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.536+0000 7f74127fc640 1 -- 172.21.15.131:0/4098789145 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74180039f0 con 0x7f741c0aff70 2023-09-12T14:55:46.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.536+0000 7f7423d35640 1 -- 172.21.15.131:0/4098789145 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f741c147d10 con 0x7f741c0aff70 2023-09-12T14:55:46.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.536+0000 7f7423d35640 1 -- 172.21.15.131:0/4098789145 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f741c06d720 con 0x7f741c0aff70 2023-09-12T14:55:46.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.536+0000 7f74127fc640 1 -- 172.21.15.131:0/4098789145 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f7418003b50 con 0x7f741c0aff70 2023-09-12T14:55:46.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.536+0000 7f74127fc640 1 --2- 172.21.15.131:0/4098789145 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7400065b20 0x7f7400067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:46.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.536+0000 7f74127fc640 1 -- 172.21.15.131:0/4098789145 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f74180aa070 con 0x7f741c0aff70 2023-09-12T14:55:46.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.536+0000 7f74212d0640 1 --2- 172.21.15.131:0/4098789145 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7400065b20 0x7f7400067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:46.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.536+0000 7f74212d0640 1 --2- 172.21.15.131:0/4098789145 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7400065b20 0x7f7400067fe0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f741c153990 tx=0x7f740c009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:46.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.540+0000 7f74127fc640 1 -- 172.21.15.131:0/4098789145 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f74180ec030 con 0x7f741c0aff70 2023-09-12T14:55:46.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.696+0000 7f7423d35640 1 -- 172.21.15.131:0/4098789145 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f741c149ec0 con 0x7f741c0aff70 2023-09-12T14:55:46.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.700+0000 7f74127fc640 1 -- 172.21.15.131:0/4098789145 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16791 (secure 0 0 0) 0x7f74180ef650 con 0x7f741c0aff70 2023-09-12T14:55:46.702 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:46.702 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":22,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:55:39.097847+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:54:55.689983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-12T14:54:57.029447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:55:46.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.700+0000 7f7423d35640 1 -- 172.21.15.131:0/4098789145 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7400065b20 msgr2=0x7f7400067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:46.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.700+0000 7f7423d35640 1 --2- 172.21.15.131:0/4098789145 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7400065b20 0x7f7400067fe0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f741c153990 tx=0x7f740c009040 comp rx=0 tx=0).stop 2023-09-12T14:55:46.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.700+0000 7f7423d35640 1 -- 172.21.15.131:0/4098789145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f741c0aff70 msgr2=0x7f741c145fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:46.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.700+0000 7f7423d35640 1 --2- 172.21.15.131:0/4098789145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f741c0aff70 0x7f741c145fa0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f741801af30 tx=0x7f74180a6780 comp rx=0 tx=0).stop 2023-09-12T14:55:46.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.700+0000 7f7423d35640 1 -- 172.21.15.131:0/4098789145 shutdown_connections 2023-09-12T14:55:46.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.700+0000 7f7423d35640 1 --2- 172.21.15.131:0/4098789145 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7400065b20 0x7f7400067fe0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.700+0000 7f7423d35640 1 --2- 172.21.15.131:0/4098789145 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f741c0cc620 0x7f741c156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.700+0000 7f7423d35640 1 --2- 172.21.15.131:0/4098789145 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f741c1553b0 0x7f741c157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.700+0000 7f7423d35640 1 --2- 172.21.15.131:0/4098789145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f741c0aff70 0x7f741c145fa0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.700+0000 7f7423d35640 1 -- 172.21.15.131:0/4098789145 >> 172.21.15.131:0/4098789145 conn(0x7f741c06a520 msgr2=0x7f741c0b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:46.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.700+0000 7f7423d35640 1 -- 172.21.15.131:0/4098789145 shutdown_connections 2023-09-12T14:55:46.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.700+0000 7f7423d35640 1 -- 172.21.15.131:0/4098789145 wait complete. 2023-09-12T14:55:46.721 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-12T14:55:46.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.836+0000 7fce958f8640 1 -- 172.21.15.131:0/139521560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce90158d70 msgr2=0x7fce9006b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:46.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.836+0000 7fce958f8640 1 --2- 172.21.15.131:0/139521560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce90158d70 0x7fce9006b430 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fce80009900 tx=0x7fce8001c830 comp rx=0 tx=0).stop 2023-09-12T14:55:46.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.836+0000 7fce958f8640 1 -- 172.21.15.131:0/139521560 shutdown_connections 2023-09-12T14:55:46.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.836+0000 7fce958f8640 1 --2- 172.21.15.131:0/139521560 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce900abae0 0x7fce900c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.836+0000 7fce958f8640 1 --2- 172.21.15.131:0/139521560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce90158d70 0x7fce9006b430 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.836+0000 7fce958f8640 1 -- 172.21.15.131:0/139521560 >> 172.21.15.131:0/139521560 conn(0x7fce90069840 msgr2=0x7fce900cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:46.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.836+0000 7fce958f8640 1 -- 172.21.15.131:0/139521560 shutdown_connections 2023-09-12T14:55:46.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.836+0000 7fce958f8640 1 -- 172.21.15.131:0/139521560 wait complete. 2023-09-12T14:55:46.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.836+0000 7fce958f8640 1 Processor -- start 2023-09-12T14:55:46.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.836+0000 7fce958f8640 1 -- start start 2023-09-12T14:55:46.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.840+0000 7fce958f8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce900abae0 0x7fce9014edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:46.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.840+0000 7fce958f8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce900cc430 0x7fce900aab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:46.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.840+0000 7fce958f8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce90158d70 0x7fce901455e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:46.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.840+0000 7fce958f8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce90156d20 con 0x7fce90158d70 2023-09-12T14:55:46.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.840+0000 7fce958f8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce90157560 con 0x7fce900abae0 2023-09-12T14:55:46.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.840+0000 7fce958f8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce9014f8f0 con 0x7fce900cc430 2023-09-12T14:55:46.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.840+0000 7fce8f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce90158d70 0x7fce901455e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:46.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.840+0000 7fce8f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce90158d70 0x7fce901455e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49082/0 (socket says 172.21.15.131:49082) 2023-09-12T14:55:46.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.840+0000 7fce8f7fe640 1 -- 172.21.15.131:0/1066238015 learned_addr learned my addr 172.21.15.131:0/1066238015 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:46.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.840+0000 7fce8effd640 1 --2- 172.21.15.131:0/1066238015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce900abae0 0x7fce9014edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:46.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.840+0000 7fce8f7fe640 1 -- 172.21.15.131:0/1066238015 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce900cc430 msgr2=0x7fce900aab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:46.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.840+0000 7fce8f7fe640 1 --2- 172.21.15.131:0/1066238015 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce900cc430 0x7fce900aab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.840+0000 7fce8f7fe640 1 -- 172.21.15.131:0/1066238015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce900abae0 msgr2=0x7fce9014edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:46.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.840+0000 7fce8f7fe640 1 --2- 172.21.15.131:0/1066238015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce900abae0 0x7fce9014edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:46.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.840+0000 7fce8f7fe640 1 -- 172.21.15.131:0/1066238015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce80009560 con 0x7fce90158d70 2023-09-12T14:55:46.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.840+0000 7fce8f7fe640 1 --2- 172.21.15.131:0/1066238015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce90158d70 0x7fce901455e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fce8800c770 tx=0x7fce8800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:46.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.840+0000 7fce87fff640 1 -- 172.21.15.131:0/1066238015 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce88003b80 con 0x7fce90158d70 2023-09-12T14:55:46.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.840+0000 7fce87fff640 1 -- 172.21.15.131:0/1066238015 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fce88003ce0 con 0x7fce90158d70 2023-09-12T14:55:46.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.840+0000 7fce958f8640 1 -- 172.21.15.131:0/1066238015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce90156f50 con 0x7fce90158d70 2023-09-12T14:55:46.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.840+0000 7fce87fff640 1 -- 172.21.15.131:0/1066238015 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8800aab0 con 0x7fce90158d70 2023-09-12T14:55:46.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.840+0000 7fce958f8640 1 -- 172.21.15.131:0/1066238015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce901571f0 con 0x7fce90158d70 2023-09-12T14:55:46.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.840+0000 7fce958f8640 1 -- 172.21.15.131:0/1066238015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce901576d0 con 0x7fce90158d70 2023-09-12T14:55:46.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.840+0000 7fce87fff640 1 -- 172.21.15.131:0/1066238015 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fce8800ac10 con 0x7fce90158d70 2023-09-12T14:55:46.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.840+0000 7fce87fff640 1 --2- 172.21.15.131:0/1066238015 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fce70065b50 0x7fce70068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:46.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.840+0000 7fce87fff640 1 -- 172.21.15.131:0/1066238015 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fce880832c0 con 0x7fce90158d70 2023-09-12T14:55:46.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.840+0000 7fce8effd640 1 --2- 172.21.15.131:0/1066238015 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fce70065b50 0x7fce70068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:46.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.840+0000 7fce8effd640 1 --2- 172.21.15.131:0/1066238015 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fce70065b50 0x7fce70068010 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fce800af000 tx=0x7fce8001b070 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:46.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:46.844+0000 7fce87fff640 1 -- 172.21.15.131:0/1066238015 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fce88050b90 con 0x7fce90158d70 2023-09-12T14:55:47.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.016+0000 7fce958f8640 1 -- 172.21.15.131:0/1066238015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fce9014f360 con 0x7fce90158d70 2023-09-12T14:55:47.022 INFO:tasks.ceph.mon.a.smithi131.stderr:2023-09-12T14:55:47.020+0000 7fdecf213640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-12T14:55:47.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.020+0000 7fce87fff640 1 -- 172.21.15.131:0/1066238015 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) v1 ==== 161+0+0 (secure 0 0 0) 0x7fce880541b0 con 0x7fce90158d70 2023-09-12T14:55:47.025 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-12T14:55:47.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.024+0000 7fce958f8640 1 -- 172.21.15.131:0/1066238015 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fce70065b50 msgr2=0x7fce70068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:47.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.024+0000 7fce958f8640 1 --2- 172.21.15.131:0/1066238015 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fce70065b50 0x7fce70068010 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fce800af000 tx=0x7fce8001b070 comp rx=0 tx=0).stop 2023-09-12T14:55:47.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.024+0000 7fce958f8640 1 -- 172.21.15.131:0/1066238015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce90158d70 msgr2=0x7fce901455e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:47.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.024+0000 7fce958f8640 1 --2- 172.21.15.131:0/1066238015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce90158d70 0x7fce901455e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fce8800c770 tx=0x7fce8800cc30 comp rx=0 tx=0).stop 2023-09-12T14:55:47.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.024+0000 7fce958f8640 1 -- 172.21.15.131:0/1066238015 shutdown_connections 2023-09-12T14:55:47.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.024+0000 7fce958f8640 1 --2- 172.21.15.131:0/1066238015 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fce70065b50 0x7fce70068010 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:47.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.024+0000 7fce958f8640 1 --2- 172.21.15.131:0/1066238015 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce900cc430 0x7fce900aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:47.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.024+0000 7fce958f8640 1 --2- 172.21.15.131:0/1066238015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce90158d70 0x7fce901455e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:47.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.024+0000 7fce958f8640 1 --2- 172.21.15.131:0/1066238015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce900abae0 0x7fce9014edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:47.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.024+0000 7fce958f8640 1 -- 172.21.15.131:0/1066238015 >> 172.21.15.131:0/1066238015 conn(0x7fce90069840 msgr2=0x7fce900af000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:47.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.024+0000 7fce958f8640 1 -- 172.21.15.131:0/1066238015 shutdown_connections 2023-09-12T14:55:47.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.024+0000 7fce958f8640 1 -- 172.21.15.131:0/1066238015 wait complete. 2023-09-12T14:55:47.038 INFO:tasks.ceph.mds.d.smithi187.stderr: -2887> 2023-09-12T14:55:02.214+0000 7f4a64e4e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-12T14:55:47.043 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-09-12T14:55:47.100 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2023-09-12T14:55:47.166 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c7caa640 1 -- 172.21.15.131:0/3072163667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84ac003980 msgr2=0x7f84ac003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:47.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c7caa640 1 --2- 172.21.15.131:0/3072163667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84ac003980 0x7f84ac003da0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f84bc009eb0 tx=0x7f84bc01d510 comp rx=0 tx=0).stop 2023-09-12T14:55:47.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c6247640 1 -- 172.21.15.131:0/3072163667 reap_dead start 2023-09-12T14:55:47.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c7caa640 1 -- 172.21.15.131:0/3072163667 shutdown_connections 2023-09-12T14:55:47.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c7caa640 1 -- 172.21.15.131:0/3072163667 >> 172.21.15.131:0/3072163667 conn(0x7f84c00cdd80 msgr2=0x7f84c00ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:47.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c7caa640 1 -- 172.21.15.131:0/3072163667 shutdown_connections 2023-09-12T14:55:47.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c7caa640 1 -- 172.21.15.131:0/3072163667 wait complete. 2023-09-12T14:55:47.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c7caa640 1 Processor -- start 2023-09-12T14:55:47.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c7caa640 1 -- start start 2023-09-12T14:55:47.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c7caa640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84c014fc60 0x7f84c015cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:47.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c7caa640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84c0150040 0x7f84c015a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:47.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c7caa640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84c015d2e0 0x7f84c015a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:47.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c7caa640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c015d7b0 con 0x7f84c0150040 2023-09-12T14:55:47.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c7caa640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c015da90 con 0x7f84c014fc60 2023-09-12T14:55:47.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c7caa640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c015dd70 con 0x7f84c015d2e0 2023-09-12T14:55:47.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c5245640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84c0150040 0x7f84c015a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:47.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c5245640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84c0150040 0x7f84c015a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49092/0 (socket says 172.21.15.131:49092) 2023-09-12T14:55:47.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c5245640 1 -- 172.21.15.131:0/2946660893 learned_addr learned my addr 172.21.15.131:0/2946660893 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:47.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c5a46640 1 --2- 172.21.15.131:0/2946660893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84c014fc60 0x7f84c015cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:47.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c6247640 1 --2- 172.21.15.131:0/2946660893 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84c015d2e0 0x7f84c015a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:47.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c5245640 1 -- 172.21.15.131:0/2946660893 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84c015d2e0 msgr2=0x7f84c015a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:47.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c5245640 1 --2- 172.21.15.131:0/2946660893 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84c015d2e0 0x7f84c015a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:47.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c5245640 1 -- 172.21.15.131:0/2946660893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84c014fc60 msgr2=0x7f84c015cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:47.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c5245640 1 --2- 172.21.15.131:0/2946660893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84c014fc60 0x7f84c015cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:47.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c5245640 1 -- 172.21.15.131:0/2946660893 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c015e050 con 0x7f84c0150040 2023-09-12T14:55:47.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c5a46640 1 --2- 172.21.15.131:0/2946660893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84c014fc60 0x7f84c015cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:55:47.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c5245640 1 --2- 172.21.15.131:0/2946660893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84c0150040 0x7f84c015a440 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f84b0006640 tx=0x7f84b00039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:47.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84b67fc640 1 -- 172.21.15.131:0/2946660893 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b000f960 con 0x7f84c0150040 2023-09-12T14:55:47.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84b67fc640 1 -- 172.21.15.131:0/2946660893 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f84b000fac0 con 0x7f84c0150040 2023-09-12T14:55:47.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c7caa640 1 -- 172.21.15.131:0/2946660893 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c015e330 con 0x7f84c0150040 2023-09-12T14:55:47.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84b67fc640 1 -- 172.21.15.131:0/2946660893 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b000feb0 con 0x7f84c0150040 2023-09-12T14:55:47.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.164+0000 7f84c7caa640 1 -- 172.21.15.131:0/2946660893 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c0069240 con 0x7f84c0150040 2023-09-12T14:55:47.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.168+0000 7f84c7caa640 1 -- 172.21.15.131:0/2946660893 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c0073580 con 0x7f84c0150040 2023-09-12T14:55:47.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.168+0000 7f84b67fc640 1 -- 172.21.15.131:0/2946660893 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f84b0011430 con 0x7f84c0150040 2023-09-12T14:55:47.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.168+0000 7f84b67fc640 1 --2- 172.21.15.131:0/2946660893 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f84a4065b20 0x7f84a4067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:47.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.168+0000 7f84b67fc640 1 -- 172.21.15.131:0/2946660893 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f84b00840a0 con 0x7f84c0150040 2023-09-12T14:55:47.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.168+0000 7f84c5a46640 1 --2- 172.21.15.131:0/2946660893 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f84a4065b20 0x7f84a4067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:47.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.168+0000 7f84c5a46640 1 --2- 172.21.15.131:0/2946660893 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f84a4065b20 0x7f84a4067fe0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f84bc0ae000 tx=0x7f84bc01a2b0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:47.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.172+0000 7f84b67fc640 1 -- 172.21.15.131:0/2946660893 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f84b0051920 con 0x7f84c0150040 2023-09-12T14:55:47.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:47.344+0000 7f84c7caa640 1 -- 172.21.15.131:0/2946660893 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f84c0070fc0 con 0x7f84c0150040 2023-09-12T14:55:48.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.024+0000 7f84b67fc640 1 -- 172.21.15.131:0/2946660893 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7f84c0070fc0 con 0x7f84c0150040 2023-09-12T14:55:48.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.028+0000 7f84c7caa640 1 -- 172.21.15.131:0/2946660893 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f84a4065b20 msgr2=0x7f84a4067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:48.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.028+0000 7f84c7caa640 1 --2- 172.21.15.131:0/2946660893 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f84a4065b20 0x7f84a4067fe0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f84bc0ae000 tx=0x7f84bc01a2b0 comp rx=0 tx=0).stop 2023-09-12T14:55:48.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.028+0000 7f84c7caa640 1 -- 172.21.15.131:0/2946660893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84c0150040 msgr2=0x7f84c015a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:48.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.028+0000 7f84c7caa640 1 --2- 172.21.15.131:0/2946660893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84c0150040 0x7f84c015a440 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f84b0006640 tx=0x7f84b00039d0 comp rx=0 tx=0).stop 2023-09-12T14:55:48.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.028+0000 7f84c7caa640 1 -- 172.21.15.131:0/2946660893 shutdown_connections 2023-09-12T14:55:48.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.028+0000 7f84c7caa640 1 --2- 172.21.15.131:0/2946660893 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f84a4065b20 0x7f84a4067fe0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:48.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.028+0000 7f84c7caa640 1 --2- 172.21.15.131:0/2946660893 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84c015d2e0 0x7f84c015a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:48.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.028+0000 7f84c7caa640 1 --2- 172.21.15.131:0/2946660893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84c0150040 0x7f84c015a440 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:48.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.028+0000 7f84c7caa640 1 --2- 172.21.15.131:0/2946660893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84c014fc60 0x7f84c015cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:48.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.028+0000 7f84c7caa640 1 -- 172.21.15.131:0/2946660893 >> 172.21.15.131:0/2946660893 conn(0x7f84c00cdd80 msgr2=0x7f84c00b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:48.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.028+0000 7f84c7caa640 1 -- 172.21.15.131:0/2946660893 shutdown_connections 2023-09-12T14:55:48.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.028+0000 7f84c7caa640 1 -- 172.21.15.131:0/2946660893 wait complete. 2023-09-12T14:55:48.051 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-09-12T14:55:48.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.184+0000 7f279235a640 1 -- 172.21.15.131:0/1794530868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f278c06b050 msgr2=0x7f278c06b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:48.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.184+0000 7f279235a640 1 --2- 172.21.15.131:0/1794530868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f278c06b050 0x7f278c06b4e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2784009c80 tx=0x7f278401cd50 comp rx=0 tx=0).stop 2023-09-12T14:55:48.189 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.184+0000 7f279235a640 1 -- 172.21.15.131:0/1794530868 shutdown_connections 2023-09-12T14:55:48.189 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.184+0000 7f279235a640 1 --2- 172.21.15.131:0/1794530868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f278c06b050 0x7f278c06b4e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:48.189 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.184+0000 7f279235a640 1 --2- 172.21.15.131:0/1794530868 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f278c0ab730 0x7f278c0abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:48.189 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.184+0000 7f279235a640 1 -- 172.21.15.131:0/1794530868 >> 172.21.15.131:0/1794530868 conn(0x7f278c069840 msgr2=0x7f278c0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:48.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.184+0000 7f279235a640 1 -- 172.21.15.131:0/1794530868 shutdown_connections 2023-09-12T14:55:48.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.184+0000 7f279235a640 1 -- 172.21.15.131:0/1794530868 wait complete. 2023-09-12T14:55:48.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.184+0000 7f279235a640 1 Processor -- start 2023-09-12T14:55:48.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.184+0000 7f279235a640 1 -- start start 2023-09-12T14:55:48.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f279235a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f278c06b050 0x7f278c1f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:48.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f279235a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f278c0ab730 0x7f278c1f7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:48.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f279235a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f278c158180 0x7f278c0b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:48.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f279235a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c0b2660 con 0x7f278c158180 2023-09-12T14:55:48.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f279235a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c1593e0 con 0x7f278c06b050 2023-09-12T14:55:48.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f279235a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278c159680 con 0x7f278c0ab730 2023-09-12T14:55:48.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f27908f7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f278c158180 0x7f278c0b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:48.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f27908f7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f278c158180 0x7f278c0b1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49120/0 (socket says 172.21.15.131:49120) 2023-09-12T14:55:48.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f27908f7640 1 -- 172.21.15.131:0/2513502298 learned_addr learned my addr 172.21.15.131:0/2513502298 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:48.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f278bfff640 1 --2- 172.21.15.131:0/2513502298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f278c06b050 0x7f278c1f42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:48.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f27908f7640 1 -- 172.21.15.131:0/2513502298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f278c0ab730 msgr2=0x7f278c1f7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:48.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f278b7fe640 1 --2- 172.21.15.131:0/2513502298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f278c0ab730 0x7f278c1f7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:48.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f27908f7640 1 --2- 172.21.15.131:0/2513502298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f278c0ab730 0x7f278c1f7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:48.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f27908f7640 1 -- 172.21.15.131:0/2513502298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f278c06b050 msgr2=0x7f278c1f42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:48.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f27908f7640 1 --2- 172.21.15.131:0/2513502298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f278c06b050 0x7f278c1f42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:48.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f27908f7640 1 -- 172.21.15.131:0/2513502298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27840098e0 con 0x7f278c158180 2023-09-12T14:55:48.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f278bfff640 1 --2- 172.21.15.131:0/2513502298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f278c06b050 0x7f278c1f42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:48.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f27908f7640 1 --2- 172.21.15.131:0/2513502298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f278c158180 0x7f278c0b1c10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f278401a980 tx=0x7f2784009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:48.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f2788ff9640 1 -- 172.21.15.131:0/2513502298 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27840ae830 con 0x7f278c158180 2023-09-12T14:55:48.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f2788ff9640 1 -- 172.21.15.131:0/2513502298 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f27840aea80 con 0x7f278c158180 2023-09-12T14:55:48.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f279235a640 1 -- 172.21.15.131:0/2513502298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f278c159960 con 0x7f278c158180 2023-09-12T14:55:48.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f2788ff9640 1 -- 172.21.15.131:0/2513502298 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27840aee50 con 0x7f278c158180 2023-09-12T14:55:48.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f279235a640 1 -- 172.21.15.131:0/2513502298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f278c1f7540 con 0x7f278c158180 2023-09-12T14:55:48.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f279235a640 1 -- 172.21.15.131:0/2513502298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f278c159e80 con 0x7f278c158180 2023-09-12T14:55:48.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f2788ff9640 1 -- 172.21.15.131:0/2513502298 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f27840b4040 con 0x7f278c158180 2023-09-12T14:55:48.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f2788ff9640 1 --2- 172.21.15.131:0/2513502298 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2778065b20 0x7f2778067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:48.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f2788ff9640 1 -- 172.21.15.131:0/2513502298 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f278411e7e0 con 0x7f278c158180 2023-09-12T14:55:48.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f278bfff640 1 --2- 172.21.15.131:0/2513502298 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2778065b20 0x7f2778067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:48.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.188+0000 7f278bfff640 1 --2- 172.21.15.131:0/2513502298 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2778065b20 0x7f2778067fe0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2780005fd0 tx=0x7f2780005e60 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:48.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.192+0000 7f2788ff9640 1 -- 172.21.15.131:0/2513502298 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f278c159e80 con 0x7f278c158180 2023-09-12T14:55:48.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:48.336+0000 7f279235a640 1 -- 172.21.15.131:0/2513502298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f278c159e80 con 0x7f278c158180 2023-09-12T14:55:49.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.180+0000 7f2788ff9640 1 -- 172.21.15.131:0/2513502298 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7f278c159e80 con 0x7f278c158180 2023-09-12T14:55:49.184 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2023-09-12T14:55:49.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.184+0000 7f279235a640 1 -- 172.21.15.131:0/2513502298 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2778065b20 msgr2=0x7f2778067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:49.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.184+0000 7f279235a640 1 --2- 172.21.15.131:0/2513502298 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2778065b20 0x7f2778067fe0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2780005fd0 tx=0x7f2780005e60 comp rx=0 tx=0).stop 2023-09-12T14:55:49.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.184+0000 7f279235a640 1 -- 172.21.15.131:0/2513502298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f278c158180 msgr2=0x7f278c0b1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:49.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.184+0000 7f279235a640 1 --2- 172.21.15.131:0/2513502298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f278c158180 0x7f278c0b1c10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f278401a980 tx=0x7f2784009390 comp rx=0 tx=0).stop 2023-09-12T14:55:49.189 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.184+0000 7f279235a640 1 -- 172.21.15.131:0/2513502298 shutdown_connections 2023-09-12T14:55:49.189 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.184+0000 7f279235a640 1 --2- 172.21.15.131:0/2513502298 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2778065b20 0x7f2778067fe0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:49.189 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.184+0000 7f279235a640 1 --2- 172.21.15.131:0/2513502298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f278c0ab730 0x7f278c1f7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:49.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.184+0000 7f279235a640 1 --2- 172.21.15.131:0/2513502298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f278c158180 0x7f278c0b1c10 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:49.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.184+0000 7f279235a640 1 --2- 172.21.15.131:0/2513502298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f278c06b050 0x7f278c1f42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:49.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.184+0000 7f279235a640 1 -- 172.21.15.131:0/2513502298 >> 172.21.15.131:0/2513502298 conn(0x7f278c069840 msgr2=0x7f278c14b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:49.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.184+0000 7f279235a640 1 -- 172.21.15.131:0/2513502298 shutdown_connections 2023-09-12T14:55:49.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.184+0000 7f279235a640 1 -- 172.21.15.131:0/2513502298 wait complete. 2023-09-12T14:55:49.214 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-09-12T14:55:49.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.344+0000 7f9115f3e640 1 -- 172.21.15.131:0/316413335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90f80062a0 msgr2=0x7f90f800caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:49.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.344+0000 7f9115f3e640 1 --2- 172.21.15.131:0/316413335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90f80062a0 0x7f90f800caa0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f910800c5c0 tx=0x7f910801d340 comp rx=0 tx=0).stop 2023-09-12T14:55:49.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.344+0000 7f9115f3e640 1 -- 172.21.15.131:0/316413335 shutdown_connections 2023-09-12T14:55:49.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.344+0000 7f9115f3e640 1 --2- 172.21.15.131:0/316413335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90f80062a0 0x7f90f800caa0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:49.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.344+0000 7f9115f3e640 1 -- 172.21.15.131:0/316413335 >> 172.21.15.131:0/316413335 conn(0x7f9110069840 msgr2=0x7f91100cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:49.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.344+0000 7f9115f3e640 1 -- 172.21.15.131:0/316413335 shutdown_connections 2023-09-12T14:55:49.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.344+0000 7f9115f3e640 1 -- 172.21.15.131:0/316413335 wait complete. 2023-09-12T14:55:49.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.344+0000 7f9115f3e640 1 Processor -- start 2023-09-12T14:55:49.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.344+0000 7f9115f3e640 1 -- start start 2023-09-12T14:55:49.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f9115f3e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90f80062a0 0x7f911013ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:49.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f9115f3e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91101398f0 0x7f9110155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:49.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f9115f3e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9110139d70 0x7f9110156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:49.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f9115f3e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91100b0ae0 con 0x7f90f80062a0 2023-09-12T14:55:49.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f9115f3e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91100b01f0 con 0x7f9110139d70 2023-09-12T14:55:49.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f9115f3e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911013d200 con 0x7f91101398f0 2023-09-12T14:55:49.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f910f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90f80062a0 0x7f911013ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:49.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f910f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90f80062a0 0x7f911013ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50918/0 (socket says 172.21.15.131:50918) 2023-09-12T14:55:49.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f910f7fe640 1 -- 172.21.15.131:0/2542351478 learned_addr learned my addr 172.21.15.131:0/2542351478 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:49.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f910ffff640 1 --2- 172.21.15.131:0/2542351478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9110139d70 0x7f9110156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:49.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f910effd640 1 --2- 172.21.15.131:0/2542351478 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91101398f0 0x7f9110155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:49.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f910f7fe640 1 -- 172.21.15.131:0/2542351478 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91101398f0 msgr2=0x7f9110155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:49.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f910f7fe640 1 --2- 172.21.15.131:0/2542351478 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91101398f0 0x7f9110155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:49.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f910f7fe640 1 -- 172.21.15.131:0/2542351478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9110139d70 msgr2=0x7f9110156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:49.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f910f7fe640 1 --2- 172.21.15.131:0/2542351478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9110139d70 0x7f9110156110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:49.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f910f7fe640 1 -- 172.21.15.131:0/2542351478 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91100aea30 con 0x7f90f80062a0 2023-09-12T14:55:49.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f910effd640 1 --2- 172.21.15.131:0/2542351478 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91101398f0 0x7f9110155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:49.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f910f7fe640 1 --2- 172.21.15.131:0/2542351478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90f80062a0 0x7f911013ccc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9100004cf0 tx=0x7f9100001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:49.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f90f7fff640 1 -- 172.21.15.131:0/2542351478 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9100016030 con 0x7f90f80062a0 2023-09-12T14:55:49.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f90f7fff640 1 -- 172.21.15.131:0/2542351478 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9100011750 con 0x7f90f80062a0 2023-09-12T14:55:49.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f9115f3e640 1 -- 172.21.15.131:0/2542351478 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91100b0420 con 0x7f90f80062a0 2023-09-12T14:55:49.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f90f7fff640 1 -- 172.21.15.131:0/2542351478 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9100011b60 con 0x7f90f80062a0 2023-09-12T14:55:49.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f9115f3e640 1 -- 172.21.15.131:0/2542351478 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91100aed10 con 0x7f90f80062a0 2023-09-12T14:55:49.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f9115f3e640 1 -- 172.21.15.131:0/2542351478 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9110073580 con 0x7f90f80062a0 2023-09-12T14:55:49.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f90f7fff640 1 -- 172.21.15.131:0/2542351478 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f9100019070 con 0x7f90f80062a0 2023-09-12T14:55:49.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f90f7fff640 1 --2- 172.21.15.131:0/2542351478 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f90fc065b50 0x7f90fc068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:49.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f910effd640 1 --2- 172.21.15.131:0/2542351478 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f90fc065b50 0x7f90fc068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:49.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f90f7fff640 1 -- 172.21.15.131:0/2542351478 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5473+0+0 (secure 0 0 0) 0x7f9100017070 con 0x7f90f80062a0 2023-09-12T14:55:49.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.348+0000 7f910effd640 1 --2- 172.21.15.131:0/2542351478 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f90fc065b50 0x7f90fc068010 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9104005fd0 tx=0x7f9104005d40 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:49.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.352+0000 7f90f7fff640 1 -- 172.21.15.131:0/2542351478 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f9100051470 con 0x7f90f80062a0 2023-09-12T14:55:49.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:49.524+0000 7f9115f3e640 1 -- 172.21.15.131:0/2542351478 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f91100ab490 con 0x7f90f80062a0 2023-09-12T14:55:50.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.192+0000 7f90f7fff640 1 -- 172.21.15.131:0/2542351478 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9100054a90 con 0x7f90f80062a0 2023-09-12T14:55:50.197 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2023-09-12T14:55:50.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.196+0000 7f9115f3e640 1 -- 172.21.15.131:0/2542351478 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f90fc065b50 msgr2=0x7f90fc068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:50.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.196+0000 7f9115f3e640 1 --2- 172.21.15.131:0/2542351478 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f90fc065b50 0x7f90fc068010 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9104005fd0 tx=0x7f9104005d40 comp rx=0 tx=0).stop 2023-09-12T14:55:50.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.196+0000 7f9115f3e640 1 -- 172.21.15.131:0/2542351478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90f80062a0 msgr2=0x7f911013ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:50.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.196+0000 7f9115f3e640 1 --2- 172.21.15.131:0/2542351478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90f80062a0 0x7f911013ccc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9100004cf0 tx=0x7f9100001640 comp rx=0 tx=0).stop 2023-09-12T14:55:50.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.200+0000 7f9115f3e640 1 -- 172.21.15.131:0/2542351478 shutdown_connections 2023-09-12T14:55:50.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.200+0000 7f9115f3e640 1 --2- 172.21.15.131:0/2542351478 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f90fc065b50 0x7f90fc068010 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:50.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.200+0000 7f9115f3e640 1 --2- 172.21.15.131:0/2542351478 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91101398f0 0x7f9110155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:50.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.200+0000 7f9115f3e640 1 --2- 172.21.15.131:0/2542351478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9110139d70 0x7f9110156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:50.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.200+0000 7f9115f3e640 1 --2- 172.21.15.131:0/2542351478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90f80062a0 0x7f911013ccc0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:50.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.200+0000 7f9115f3e640 1 -- 172.21.15.131:0/2542351478 >> 172.21.15.131:0/2542351478 conn(0x7f9110069840 msgr2=0x7f91100cacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:50.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.200+0000 7f9115f3e640 1 -- 172.21.15.131:0/2542351478 shutdown_connections 2023-09-12T14:55:50.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.200+0000 7f9115f3e640 1 -- 172.21.15.131:0/2542351478 wait complete. 2023-09-12T14:55:50.228 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-12T14:55:50.228 INFO:tasks.ceph.mds.a:Stopping old one... 2023-09-12T14:55:50.229 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-12T14:55:50.229 INFO:teuthology.orchestra.run:waiting for 300 2023-09-12T14:55:50.229 INFO:tasks.ceph.mds.c:Restarting daemon 2023-09-12T14:55:50.229 INFO:tasks.ceph.mds.c:Stopping old one... 2023-09-12T14:55:50.230 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-09-12T14:55:50.230 INFO:teuthology.orchestra.run:waiting for 300 2023-09-12T14:55:50.230 INFO:tasks.ceph.mds.b:Restarting daemon 2023-09-12T14:55:50.231 INFO:tasks.ceph.mds.b:Stopping old one... 2023-09-12T14:55:50.231 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-09-12T14:55:50.231 INFO:teuthology.orchestra.run:waiting for 300 2023-09-12T14:55:50.232 INFO:tasks.ceph.mds.d:Restarting daemon 2023-09-12T14:55:50.232 INFO:tasks.ceph.mds.d:Stopping old one... 2023-09-12T14:55:50.232 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-09-12T14:55:50.232 INFO:teuthology.orchestra.run:waiting for 300 2023-09-12T14:55:50.240 INFO:tasks.ceph.mds.a:Stopped 2023-09-12T14:55:50.241 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-12T14:55:50.246 INFO:tasks.ceph.mds.c:Stopped 2023-09-12T14:55:50.246 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-09-12T14:55:50.247 INFO:tasks.ceph.mds.b:Stopped 2023-09-12T14:55:50.248 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-09-12T14:55:50.250 INFO:tasks.ceph.mds.d:Stopped 2023-09-12T14:55:50.251 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-09-12T14:55:50.294 INFO:tasks.ceph.mds.a:Started 2023-09-12T14:55:50.295 DEBUG:teuthology.parallel:result is None 2023-09-12T14:55:50.296 INFO:tasks.ceph.mds.b:Started 2023-09-12T14:55:50.296 DEBUG:teuthology.parallel:result is None 2023-09-12T14:55:50.297 INFO:tasks.ceph.mds.c:Started 2023-09-12T14:55:50.298 INFO:tasks.ceph.mds.d:Started 2023-09-12T14:55:50.298 DEBUG:teuthology.parallel:result is None 2023-09-12T14:55:50.298 DEBUG:teuthology.parallel:result is None 2023-09-12T14:55:50.299 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-09-12T14:55:50.354 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2023-09-12T14:55:50.355 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2023-09-12T14:55:50.362 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2023-09-12T14:55:50.363 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2023-09-12T14:55:50.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee7e3bc640 1 -- 172.21.15.131:0/435987805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee780cc620 msgr2=0x7fee78162940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:50.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee7e3bc640 1 --2- 172.21.15.131:0/435987805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee780cc620 0x7fee78162940 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fee70009bf0 tx=0x7fee7001cbc0 comp rx=0 tx=0).stop 2023-09-12T14:55:50.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee7e3bc640 1 -- 172.21.15.131:0/435987805 shutdown_connections 2023-09-12T14:55:50.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee7e3bc640 1 --2- 172.21.15.131:0/435987805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee780cc620 0x7fee78162940 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:50.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee7e3bc640 1 -- 172.21.15.131:0/435987805 >> 172.21.15.131:0/435987805 conn(0x7fee7806a520 msgr2=0x7fee7806a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:50.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee7e3bc640 1 -- 172.21.15.131:0/435987805 shutdown_connections 2023-09-12T14:55:50.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee7e3bc640 1 -- 172.21.15.131:0/435987805 wait complete. 2023-09-12T14:55:50.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee7e3bc640 1 Processor -- start 2023-09-12T14:55:50.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee7e3bc640 1 -- start start 2023-09-12T14:55:50.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee7e3bc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee780b0140 0x7fee7814d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:50.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee7e3bc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee780cc620 0x7fee78159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:50.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee7e3bc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee781553b0 0x7fee781596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:50.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee7e3bc640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee781544f0 con 0x7fee780b0140 2023-09-12T14:55:50.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee7e3bc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee78154740 con 0x7fee780cc620 2023-09-12T14:55:50.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee7e3bc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee78154a20 con 0x7fee781553b0 2023-09-12T14:55:50.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee77fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee780b0140 0x7fee7814d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:50.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee77fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee780b0140 0x7fee7814d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50966/0 (socket says 172.21.15.131:50966) 2023-09-12T14:55:50.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee77fff640 1 -- 172.21.15.131:0/626326083 learned_addr learned my addr 172.21.15.131:0/626326083 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:50.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee7c959640 1 --2- 172.21.15.131:0/626326083 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee781553b0 0x7fee781596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:50.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee777fe640 1 --2- 172.21.15.131:0/626326083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee780cc620 0x7fee78159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:50.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee77fff640 1 -- 172.21.15.131:0/626326083 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee781553b0 msgr2=0x7fee781596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:50.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee77fff640 1 --2- 172.21.15.131:0/626326083 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee781553b0 0x7fee781596d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:50.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee77fff640 1 -- 172.21.15.131:0/626326083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee780cc620 msgr2=0x7fee78159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:50.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee77fff640 1 --2- 172.21.15.131:0/626326083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee780cc620 0x7fee78159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:50.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee77fff640 1 -- 172.21.15.131:0/626326083 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee70009820 con 0x7fee780b0140 2023-09-12T14:55:50.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee77fff640 1 --2- 172.21.15.131:0/626326083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee780b0140 0x7fee7814d470 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fee6800c370 tx=0x7fee6800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:50.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee74ff9640 1 -- 172.21.15.131:0/626326083 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee680052d0 con 0x7fee780b0140 2023-09-12T14:55:50.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee74ff9640 1 -- 172.21.15.131:0/626326083 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fee68007020 con 0x7fee780b0140 2023-09-12T14:55:50.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee74ff9640 1 -- 172.21.15.131:0/626326083 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee68007430 con 0x7fee780b0140 2023-09-12T14:55:50.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee7e3bc640 1 -- 172.21.15.131:0/626326083 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee78153490 con 0x7fee780b0140 2023-09-12T14:55:50.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.420+0000 7fee7e3bc640 1 -- 172.21.15.131:0/626326083 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee7814d9b0 con 0x7fee780b0140 2023-09-12T14:55:50.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.424+0000 7fee7e3bc640 1 -- 172.21.15.131:0/626326083 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee7806d720 con 0x7fee780b0140 2023-09-12T14:55:50.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.424+0000 7fee74ff9640 1 -- 172.21.15.131:0/626326083 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fee68007590 con 0x7fee780b0140 2023-09-12T14:55:50.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.424+0000 7fee74ff9640 1 --2- 172.21.15.131:0/626326083 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fee64065b20 0x7fee64067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:50.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.424+0000 7fee74ff9640 1 -- 172.21.15.131:0/626326083 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fee68082ae0 con 0x7fee780b0140 2023-09-12T14:55:50.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.424+0000 7fee777fe640 1 --2- 172.21.15.131:0/626326083 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fee64065b20 0x7fee64067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:50.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.424+0000 7fee777fe640 1 --2- 172.21.15.131:0/626326083 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fee64065b20 0x7fee64067fe0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fee7814e1b0 tx=0x7fee6c009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:50.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.428+0000 7fee74ff9640 1 -- 172.21.15.131:0/626326083 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fee680507c0 con 0x7fee780b0140 2023-09-12T14:55:50.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:50.600+0000 7fee7e3bc640 1 -- 172.21.15.131:0/626326083 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fee78154c60 con 0x7fee780b0140 2023-09-12T14:55:51.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.228+0000 7fee74ff9640 1 -- 172.21.15.131:0/626326083 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) v1 ==== 97+0+0 (secure 0 0 0) 0x7fee78154c60 con 0x7fee780b0140 2023-09-12T14:55:51.234 INFO:teuthology.orchestra.run.smithi131.stderr: removed all blocklist entries 2023-09-12T14:55:51.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.232+0000 7fee7e3bc640 1 -- 172.21.15.131:0/626326083 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fee64065b20 msgr2=0x7fee64067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:51.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.232+0000 7fee7e3bc640 1 --2- 172.21.15.131:0/626326083 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fee64065b20 0x7fee64067fe0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fee7814e1b0 tx=0x7fee6c009040 comp rx=0 tx=0).stop 2023-09-12T14:55:51.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.232+0000 7fee7e3bc640 1 -- 172.21.15.131:0/626326083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee780b0140 msgr2=0x7fee7814d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:51.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.232+0000 7fee7e3bc640 1 --2- 172.21.15.131:0/626326083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee780b0140 0x7fee7814d470 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fee6800c370 tx=0x7fee6800c830 comp rx=0 tx=0).stop 2023-09-12T14:55:51.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.232+0000 7fee7e3bc640 1 -- 172.21.15.131:0/626326083 shutdown_connections 2023-09-12T14:55:51.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.232+0000 7fee7e3bc640 1 --2- 172.21.15.131:0/626326083 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fee64065b20 0x7fee64067fe0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:51.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.232+0000 7fee7e3bc640 1 --2- 172.21.15.131:0/626326083 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee781553b0 0x7fee781596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:51.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.232+0000 7fee7e3bc640 1 --2- 172.21.15.131:0/626326083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee780cc620 0x7fee78159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:51.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.232+0000 7fee7e3bc640 1 --2- 172.21.15.131:0/626326083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee780b0140 0x7fee7814d470 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:51.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.232+0000 7fee7e3bc640 1 -- 172.21.15.131:0/626326083 >> 172.21.15.131:0/626326083 conn(0x7fee7806a520 msgr2=0x7fee7815a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:51.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.232+0000 7fee7e3bc640 1 -- 172.21.15.131:0/626326083 shutdown_connections 2023-09-12T14:55:51.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.232+0000 7fee7e3bc640 1 -- 172.21.15.131:0/626326083 wait complete. 2023-09-12T14:55:51.263 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-09-12T14:55:51.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad70c2b640 1 -- 172.21.15.131:0/3528568520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad58003980 msgr2=0x7fad58003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:51.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad70c2b640 1 --2- 172.21.15.131:0/3528568520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad58003980 0x7fad58003da0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0016b0 tx=0x7fad5c01c750 comp rx=0 tx=0).stop 2023-09-12T14:55:51.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad70c2b640 1 -- 172.21.15.131:0/3528568520 shutdown_connections 2023-09-12T14:55:51.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad70c2b640 1 --2- 172.21.15.131:0/3528568520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad58003980 0x7fad58003da0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:51.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad70c2b640 1 -- 172.21.15.131:0/3528568520 >> 172.21.15.131:0/3528568520 conn(0x7fad6c069840 msgr2=0x7fad6c0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:51.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad70c2b640 1 -- 172.21.15.131:0/3528568520 shutdown_connections 2023-09-12T14:55:51.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad70c2b640 1 -- 172.21.15.131:0/3528568520 wait complete. 2023-09-12T14:55:51.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad70c2b640 1 Processor -- start 2023-09-12T14:55:51.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad70c2b640 1 -- start start 2023-09-12T14:55:51.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad70c2b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad58003980 0x7fad6c1410d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:51.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad70c2b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad6c14bad0 0x7fad6c14bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:51.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad70c2b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad6c14ed40 0x7fad6c14f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:51.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad70c2b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c14e920 con 0x7fad6c14ed40 2023-09-12T14:55:51.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad70c2b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c14c870 con 0x7fad58003980 2023-09-12T14:55:51.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad70c2b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c14cb50 con 0x7fad6c14bad0 2023-09-12T14:55:51.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad6ad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad6c14ed40 0x7fad6c14f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:51.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad6ad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad6c14ed40 0x7fad6c14f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50986/0 (socket says 172.21.15.131:50986) 2023-09-12T14:55:51.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad6ad9d640 1 -- 172.21.15.131:0/2523369705 learned_addr learned my addr 172.21.15.131:0/2523369705 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:51.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad63fff640 1 --2- 172.21.15.131:0/2523369705 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad6c14bad0 0x7fad6c14bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:51.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad6a59c640 1 --2- 172.21.15.131:0/2523369705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad58003980 0x7fad6c1410d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:51.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad6ad9d640 1 -- 172.21.15.131:0/2523369705 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad6c14bad0 msgr2=0x7fad6c14bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:51.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad6ad9d640 1 --2- 172.21.15.131:0/2523369705 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad6c14bad0 0x7fad6c14bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:51.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad6ad9d640 1 -- 172.21.15.131:0/2523369705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad58003980 msgr2=0x7fad6c1410d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:51.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad6ad9d640 1 --2- 172.21.15.131:0/2523369705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad58003980 0x7fad6c1410d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:51.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad6ad9d640 1 -- 172.21.15.131:0/2523369705 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6c14ce30 con 0x7fad6c14ed40 2023-09-12T14:55:51.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad6a59c640 1 --2- 172.21.15.131:0/2523369705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad58003980 0x7fad6c1410d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:51.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad6ad9d640 1 --2- 172.21.15.131:0/2523369705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad6c14ed40 0x7fad6c14f1c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fad6c070940 tx=0x7fad64009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:51.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad62ffd640 1 -- 172.21.15.131:0/2523369705 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad64018070 con 0x7fad6c14ed40 2023-09-12T14:55:51.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad62ffd640 1 -- 172.21.15.131:0/2523369705 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fad64004c20 con 0x7fad6c14ed40 2023-09-12T14:55:51.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad70c2b640 1 -- 172.21.15.131:0/2523369705 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad6c14d110 con 0x7fad6c14ed40 2023-09-12T14:55:51.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.392+0000 7fad62ffd640 1 -- 172.21.15.131:0/2523369705 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad640106a0 con 0x7fad6c14ed40 2023-09-12T14:55:51.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.396+0000 7fad70c2b640 1 -- 172.21.15.131:0/2523369705 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad6c141610 con 0x7fad6c14ed40 2023-09-12T14:55:51.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.396+0000 7fad70c2b640 1 -- 172.21.15.131:0/2523369705 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad6c14ea60 con 0x7fad6c14ed40 2023-09-12T14:55:51.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.396+0000 7fad62ffd640 1 -- 172.21.15.131:0/2523369705 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fad64010800 con 0x7fad6c14ed40 2023-09-12T14:55:51.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.396+0000 7fad62ffd640 1 --2- 172.21.15.131:0/2523369705 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fad4c065b20 0x7fad4c067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:51.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.396+0000 7fad6a59c640 1 --2- 172.21.15.131:0/2523369705 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fad4c065b20 0x7fad4c067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:51.405 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.396+0000 7fad62ffd640 1 -- 172.21.15.131:0/2523369705 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fad64083150 con 0x7fad6c14ed40 2023-09-12T14:55:51.405 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.396+0000 7fad6a59c640 1 --2- 172.21.15.131:0/2523369705 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fad4c065b20 0x7fad4c067fe0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fad5c001680 tx=0x7fad5c0a8040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:51.405 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.400+0000 7fad62ffd640 1 -- 172.21.15.131:0/2523369705 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fad6c14ea60 con 0x7fad6c14ed40 2023-09-12T14:55:51.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.608+0000 7fad70c2b640 1 -- 172.21.15.131:0/2523369705 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fad6c073580 con 0x7fad6c14ed40 2023-09-12T14:55:51.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.608+0000 7fad62ffd640 1 -- 172.21.15.131:0/2523369705 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fad64050d60 con 0x7fad6c14ed40 2023-09-12T14:55:51.611 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:51.611 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-09-12T14:55:51.611 INFO:teuthology.orchestra.run.smithi131.stdout: "auth_dump": [ 2023-09-12T14:55:51.612 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-09-12T14:55:51.612 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.a", 2023-09-12T14:55:51.612 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQCFewBl66m1IBAAACThNe9wpTrHBN/ZY493xw==", 2023-09-12T14:55:51.612 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-09-12T14:55:51.613 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-09-12T14:55:51.613 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-09-12T14:55:51.613 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-09-12T14:55:51.613 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-09-12T14:55:51.613 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:51.614 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:51.614 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-09-12T14:55:51.614 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.b", 2023-09-12T14:55:51.614 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQCFewBlOENULxAAlYD82hfE/VldCdcujkgyZg==", 2023-09-12T14:55:51.614 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-09-12T14:55:51.615 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-09-12T14:55:51.615 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-09-12T14:55:51.615 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-09-12T14:55:51.615 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-09-12T14:55:51.615 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:51.616 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:51.616 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-09-12T14:55:51.616 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.c", 2023-09-12T14:55:51.616 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQCFewBlQAVaKRAA+H71HHnLvjttM1qDGTfy+g==", 2023-09-12T14:55:51.616 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-09-12T14:55:51.616 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-09-12T14:55:51.617 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-09-12T14:55:51.617 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-09-12T14:55:51.617 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-09-12T14:55:51.617 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:51.617 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:51.618 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-09-12T14:55:51.618 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.d", 2023-09-12T14:55:51.618 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQCFewBlE6u5NxAAp04RwzbaDxHv+CD99Q6fyQ==", 2023-09-12T14:55:51.618 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-09-12T14:55:51.618 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-09-12T14:55:51.619 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-09-12T14:55:51.619 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-09-12T14:55:51.619 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-09-12T14:55:51.619 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:51.619 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:51.620 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-09-12T14:55:51.620 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.0", 2023-09-12T14:55:51.620 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQCJewBl6FWSCBAAh9FESSSN2PT1cHiFqDJC3Q==", 2023-09-12T14:55:51.620 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-09-12T14:55:51.621 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-09-12T14:55:51.621 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-09-12T14:55:51.621 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-09-12T14:55:51.621 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:51.621 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:51.621 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-09-12T14:55:51.622 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.1", 2023-09-12T14:55:51.622 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQCMewBliigYDxAA7a6XlL1w97mUPNj7kuqADA==", 2023-09-12T14:55:51.622 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-09-12T14:55:51.622 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-09-12T14:55:51.622 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-09-12T14:55:51.623 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-09-12T14:55:51.623 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:51.623 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:51.623 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-09-12T14:55:51.623 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.2", 2023-09-12T14:55:51.624 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQCPewBlAjd9FxAA3sCdZikpCaVw/JlslRb2VQ==", 2023-09-12T14:55:51.624 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-09-12T14:55:51.624 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-09-12T14:55:51.624 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-09-12T14:55:51.624 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-09-12T14:55:51.625 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:51.625 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:51.625 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-09-12T14:55:51.625 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.3", 2023-09-12T14:55:51.625 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQCSewBlr8HWHxAAj7UuvuDHcavTL0kZ6KIvqw==", 2023-09-12T14:55:51.626 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-09-12T14:55:51.626 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-09-12T14:55:51.626 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-09-12T14:55:51.626 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-09-12T14:55:51.626 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:51.627 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:51.627 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-09-12T14:55:51.627 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.4", 2023-09-12T14:55:51.627 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQCYewBlRZ5qHhAAjba2RMH0atRum97TUk8ZbQ==", 2023-09-12T14:55:51.627 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-09-12T14:55:51.628 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-09-12T14:55:51.628 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-09-12T14:55:51.628 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-09-12T14:55:51.628 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:51.630 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:51.630 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-09-12T14:55:51.630 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.5", 2023-09-12T14:55:51.631 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQCbewBlQv0QJRAAg4wipYCjrBl3Lda0R2F3bQ==", 2023-09-12T14:55:51.631 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-09-12T14:55:51.631 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-09-12T14:55:51.632 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-09-12T14:55:51.632 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-09-12T14:55:51.632 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:51.632 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:51.633 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-09-12T14:55:51.633 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.6", 2023-09-12T14:55:51.633 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQCeewBlTf8zLxAAHlx1RrJ724NUYMWb+C4Viw==", 2023-09-12T14:55:51.633 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-09-12T14:55:51.634 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-09-12T14:55:51.634 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-09-12T14:55:51.634 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-09-12T14:55:51.634 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:51.635 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:51.635 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-09-12T14:55:51.635 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.7", 2023-09-12T14:55:51.635 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQChewBlEb4aNRAAB9VRHeTwK6TnSxggszvCUw==", 2023-09-12T14:55:51.635 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-09-12T14:55:51.636 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-09-12T14:55:51.636 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-09-12T14:55:51.636 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-09-12T14:55:51.636 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:51.636 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:51.637 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-09-12T14:55:51.637 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.0", 2023-09-12T14:55:51.637 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQCGewBlR6eoARAACciqMiYRB92w6zQMKfJGiA==", 2023-09-12T14:55:51.638 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-09-12T14:55:51.638 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-09-12T14:55:51.638 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-09-12T14:55:51.638 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-09-12T14:55:51.639 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-09-12T14:55:51.639 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:51.639 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:51.639 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-09-12T14:55:51.640 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.1", 2023-09-12T14:55:51.640 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQCGewBl6JxxCRAAe0pokP2AETTh2dX4lkCBmw==", 2023-09-12T14:55:51.640 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-09-12T14:55:51.640 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-09-12T14:55:51.641 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-09-12T14:55:51.641 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-09-12T14:55:51.641 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-09-12T14:55:51.642 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:51.642 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:51.642 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-09-12T14:55:51.642 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.2", 2023-09-12T14:55:51.643 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQCGewBl+Oa3BhAAHM8f/b08kfEhPZhkkySIRg==", 2023-09-12T14:55:51.643 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-09-12T14:55:51.643 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-09-12T14:55:51.643 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-09-12T14:55:51.644 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-09-12T14:55:51.644 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-09-12T14:55:51.644 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:51.644 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:51.645 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-09-12T14:55:51.645 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.3", 2023-09-12T14:55:51.645 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQCGewBl+CSgDhAA+uCVLnyp7u4XUjxC5XdkJw==", 2023-09-12T14:55:51.645 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-09-12T14:55:51.646 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-09-12T14:55:51.646 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-09-12T14:55:51.646 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-09-12T14:55:51.646 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-09-12T14:55:51.647 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:51.647 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:51.647 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-09-12T14:55:51.647 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.admin", 2023-09-12T14:55:51.648 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQCFewBlm2xGCRAAeiaP60jvmVdGWaJYn/6qdw==", 2023-09-12T14:55:51.648 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-09-12T14:55:51.648 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-09-12T14:55:51.648 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-09-12T14:55:51.648 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-09-12T14:55:51.649 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-09-12T14:55:51.649 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:51.649 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:51.649 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-09-12T14:55:51.650 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-mds", 2023-09-12T14:55:51.650 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQCoewBlxSYqCRAAZHfFiNdFbSQLMzb6GYgcvg==", 2023-09-12T14:55:51.650 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-09-12T14:55:51.650 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-mds" 2023-09-12T14:55:51.650 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:51.651 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:51.651 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-09-12T14:55:51.651 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-mgr", 2023-09-12T14:55:51.651 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQCoewBl4jAqCRAArnUe4NSQa2N3Am/drT9OLA==", 2023-09-12T14:55:51.652 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-09-12T14:55:51.652 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-mgr" 2023-09-12T14:55:51.652 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:51.652 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:51.653 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-09-12T14:55:51.653 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-osd", 2023-09-12T14:55:51.654 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQCoewBlazgqCRAA8v72xExAU3V5IS3rcr0DtQ==", 2023-09-12T14:55:51.654 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-09-12T14:55:51.655 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-osd" 2023-09-12T14:55:51.655 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:51.655 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:51.656 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-09-12T14:55:51.656 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-rbd", 2023-09-12T14:55:51.656 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQCoewBl8j8qCRAASDI+zpjYm+60m57W1PDIlA==", 2023-09-12T14:55:51.657 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-09-12T14:55:51.657 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-rbd" 2023-09-12T14:55:51.657 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:51.657 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:51.658 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-09-12T14:55:51.658 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-09-12T14:55:51.658 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQCoewBlmkcqCRAAqExQ87dmlCF2dmDXhSp42w==", 2023-09-12T14:55:51.658 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-09-12T14:55:51.659 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-09-12T14:55:51.659 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:51.659 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:51.659 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-09-12T14:55:51.659 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-rgw", 2023-09-12T14:55:51.660 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQCoewBlZE8qCRAAHhLTY9QWi8JJu4q2CwXWnA==", 2023-09-12T14:55:51.660 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-09-12T14:55:51.660 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-rgw" 2023-09-12T14:55:51.660 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:51.661 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:51.661 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-09-12T14:55:51.661 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.x", 2023-09-12T14:55:51.661 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQCFewBlX17uHBAAVt2YwHYj+9r1X8RBvmLEDA==", 2023-09-12T14:55:51.661 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-09-12T14:55:51.662 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-09-12T14:55:51.662 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2023-09-12T14:55:51.662 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-09-12T14:55:51.662 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:51.663 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:55:51.663 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-09-12T14:55:51.663 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.y", 2023-09-12T14:55:51.663 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQCFewBliZ8/GhAAAMLjw4GVY9sK0KNQe7nJ/Q==", 2023-09-12T14:55:51.663 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-09-12T14:55:51.664 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-09-12T14:55:51.664 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2023-09-12T14:55:51.664 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-09-12T14:55:51.664 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:51.664 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:51.665 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2023-09-12T14:55:51.665 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-09-12T14:55:51.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.608+0000 7fad70c2b640 1 -- 172.21.15.131:0/2523369705 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fad4c065b20 msgr2=0x7fad4c067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:51.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.608+0000 7fad70c2b640 1 --2- 172.21.15.131:0/2523369705 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fad4c065b20 0x7fad4c067fe0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fad5c001680 tx=0x7fad5c0a8040 comp rx=0 tx=0).stop 2023-09-12T14:55:51.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.608+0000 7fad70c2b640 1 -- 172.21.15.131:0/2523369705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad6c14ed40 msgr2=0x7fad6c14f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:51.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.608+0000 7fad70c2b640 1 --2- 172.21.15.131:0/2523369705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad6c14ed40 0x7fad6c14f1c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fad6c070940 tx=0x7fad64009b00 comp rx=0 tx=0).stop 2023-09-12T14:55:51.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.608+0000 7fad70c2b640 1 -- 172.21.15.131:0/2523369705 shutdown_connections 2023-09-12T14:55:51.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.608+0000 7fad70c2b640 1 --2- 172.21.15.131:0/2523369705 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fad4c065b20 0x7fad4c067fe0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:51.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.608+0000 7fad70c2b640 1 --2- 172.21.15.131:0/2523369705 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad6c14bad0 0x7fad6c14bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:51.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.608+0000 7fad70c2b640 1 --2- 172.21.15.131:0/2523369705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad6c14ed40 0x7fad6c14f1c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:51.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.608+0000 7fad70c2b640 1 --2- 172.21.15.131:0/2523369705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad58003980 0x7fad6c1410d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:51.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.608+0000 7fad70c2b640 1 -- 172.21.15.131:0/2523369705 >> 172.21.15.131:0/2523369705 conn(0x7fad6c069840 msgr2=0x7fad6c0acf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:51.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.608+0000 7fad70c2b640 1 -- 172.21.15.131:0/2523369705 shutdown_connections 2023-09-12T14:55:51.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.608+0000 7fad70c2b640 1 -- 172.21.15.131:0/2523369705 wait complete. 2023-09-12T14:55:51.670 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-09-12T14:55:51.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.796+0000 7f967d21d640 1 -- 172.21.15.131:0/1442964194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9678158180 msgr2=0x7f96780cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:51.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.796+0000 7f967d21d640 1 --2- 172.21.15.131:0/1442964194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9678158180 0x7f96780cae70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f967806cd90 tx=0x7f966c01c870 comp rx=0 tx=0).stop 2023-09-12T14:55:51.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.796+0000 7f967d21d640 1 -- 172.21.15.131:0/1442964194 shutdown_connections 2023-09-12T14:55:51.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.796+0000 7f967d21d640 1 --2- 172.21.15.131:0/1442964194 >> v2:172.21.15.187:3300/0 conn(0x7f967806f390 0x7f967806f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:51.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.796+0000 7f967d21d640 1 --2- 172.21.15.131:0/1442964194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9678158180 0x7f96780cae70 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:51.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.796+0000 7f967d21d640 1 --2- 172.21.15.131:0/1442964194 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96780ab730 0x7f96780abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:51.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.796+0000 7f967d21d640 1 -- 172.21.15.131:0/1442964194 >> 172.21.15.131:0/1442964194 conn(0x7f9678069840 msgr2=0x7f96780cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:51.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.796+0000 7f967d21d640 1 -- 172.21.15.131:0/1442964194 shutdown_connections 2023-09-12T14:55:51.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.796+0000 7f967d21d640 1 -- 172.21.15.131:0/1442964194 wait complete. 2023-09-12T14:55:51.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.796+0000 7f967d21d640 1 Processor -- start 2023-09-12T14:55:51.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.796+0000 7f967d21d640 1 -- start start 2023-09-12T14:55:51.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.796+0000 7f967d21d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f967806f390 0x7f9678145010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:51.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.796+0000 7f967d21d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96780ab730 0x7f96781478d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:51.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.796+0000 7f967d21d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9678158180 0x7f9678156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:51.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.796+0000 7f967d21d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9678159c10 con 0x7f96780ab730 2023-09-12T14:55:51.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.796+0000 7f967d21d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9678147e10 con 0x7f967806f390 2023-09-12T14:55:51.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.796+0000 7f967d21d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f967814e4f0 con 0x7f9678158180 2023-09-12T14:55:51.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.796+0000 7f967659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96780ab730 0x7f96781478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:51.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.796+0000 7f967659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96780ab730 0x7f96781478d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51006/0 (socket says 172.21.15.131:51006) 2023-09-12T14:55:51.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.796+0000 7f967659c640 1 -- 172.21.15.131:0/835771866 learned_addr learned my addr 172.21.15.131:0/835771866 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:51.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.796+0000 7f9676d9d640 1 --2- 172.21.15.131:0/835771866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f967806f390 0x7f9678145010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:51.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.796+0000 7f967759e640 1 --2- 172.21.15.131:0/835771866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9678158180 0x7f9678156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:51.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.796+0000 7f967659c640 1 -- 172.21.15.131:0/835771866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9678158180 msgr2=0x7f9678156b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:51.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.796+0000 7f967659c640 1 --2- 172.21.15.131:0/835771866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9678158180 0x7f9678156b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:51.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.796+0000 7f967659c640 1 -- 172.21.15.131:0/835771866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f967806f390 msgr2=0x7f9678145010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:51.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.796+0000 7f967659c640 1 --2- 172.21.15.131:0/835771866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f967806f390 0x7f9678145010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:51.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.796+0000 7f967659c640 1 -- 172.21.15.131:0/835771866 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f966c009560 con 0x7f96780ab730 2023-09-12T14:55:51.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.800+0000 7f967659c640 1 --2- 172.21.15.131:0/835771866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96780ab730 0x7f96781478d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f966c0b0040 tx=0x7f966c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:51.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.800+0000 7f965f7fe640 1 -- 172.21.15.131:0/835771866 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966c005230 con 0x7f96780ab730 2023-09-12T14:55:51.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.800+0000 7f965f7fe640 1 -- 172.21.15.131:0/835771866 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f966c005480 con 0x7f96780ab730 2023-09-12T14:55:51.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.800+0000 7f967d21d640 1 -- 172.21.15.131:0/835771866 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96781592e0 con 0x7f96780ab730 2023-09-12T14:55:51.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.800+0000 7f965f7fe640 1 -- 172.21.15.131:0/835771866 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966c004930 con 0x7f96780ab730 2023-09-12T14:55:51.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.800+0000 7f967d21d640 1 -- 172.21.15.131:0/835771866 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96781598a0 con 0x7f96780ab730 2023-09-12T14:55:51.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.800+0000 7f967d21d640 1 -- 172.21.15.131:0/835771866 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9678159dc0 con 0x7f96780ab730 2023-09-12T14:55:51.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.800+0000 7f965f7fe640 1 -- 172.21.15.131:0/835771866 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f966c0b4070 con 0x7f96780ab730 2023-09-12T14:55:51.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.800+0000 7f965f7fe640 1 --2- 172.21.15.131:0/835771866 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9664065b50 0x7f9664068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:51.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.800+0000 7f965f7fe640 1 -- 172.21.15.131:0/835771866 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f966c11cc70 con 0x7f96780ab730 2023-09-12T14:55:51.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.800+0000 7f9676d9d640 1 --2- 172.21.15.131:0/835771866 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9664065b50 0x7f9664068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:51.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.800+0000 7f9676d9d640 1 --2- 172.21.15.131:0/835771866 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9664065b50 0x7f9664068010 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f96700058d0 tx=0x7f9670005860 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:51.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:51.804+0000 7f965f7fe640 1 -- 172.21.15.131:0/835771866 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f9678159dc0 con 0x7f96780ab730 2023-09-12T14:55:52.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.012+0000 7f967d21d640 1 -- 172.21.15.131:0/835771866 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f96781525b0 con 0x7f96780ab730 2023-09-12T14:55:52.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.016+0000 7f965f7fe640 1 -- 172.21.15.131:0/835771866 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v15) v1 ==== 90+0+0 (secure 0 0 0) 0x7f966c0b20a0 con 0x7f96780ab730 2023-09-12T14:55:52.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.020+0000 7f967d21d640 1 -- 172.21.15.131:0/835771866 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9664065b50 msgr2=0x7f9664068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:52.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.020+0000 7f967d21d640 1 --2- 172.21.15.131:0/835771866 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9664065b50 0x7f9664068010 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f96700058d0 tx=0x7f9670005860 comp rx=0 tx=0).stop 2023-09-12T14:55:52.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.020+0000 7f967d21d640 1 -- 172.21.15.131:0/835771866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96780ab730 msgr2=0x7f96781478d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:52.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.020+0000 7f967d21d640 1 --2- 172.21.15.131:0/835771866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96780ab730 0x7f96781478d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f966c0b0040 tx=0x7f966c004600 comp rx=0 tx=0).stop 2023-09-12T14:55:52.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.020+0000 7f967d21d640 1 -- 172.21.15.131:0/835771866 shutdown_connections 2023-09-12T14:55:52.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.020+0000 7f967d21d640 1 --2- 172.21.15.131:0/835771866 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9664065b50 0x7f9664068010 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:52.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.020+0000 7f967d21d640 1 --2- 172.21.15.131:0/835771866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9678158180 0x7f9678156b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:52.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.020+0000 7f967d21d640 1 --2- 172.21.15.131:0/835771866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96780ab730 0x7f96781478d0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:52.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.020+0000 7f967d21d640 1 --2- 172.21.15.131:0/835771866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f967806f390 0x7f9678145010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:52.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.020+0000 7f967d21d640 1 -- 172.21.15.131:0/835771866 >> 172.21.15.131:0/835771866 conn(0x7f9678069840 msgr2=0x7f96780adf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:52.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.020+0000 7f967d21d640 1 -- 172.21.15.131:0/835771866 shutdown_connections 2023-09-12T14:55:52.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.020+0000 7f967d21d640 1 -- 172.21.15.131:0/835771866 wait complete. 2023-09-12T14:55:52.040 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-09-12T14:55:52.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.164+0000 7f3f5348f640 1 -- 172.21.15.131:0/1448335104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f380047d0 msgr2=0x7f3f38004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:52.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.164+0000 7f3f5348f640 1 --2- 172.21.15.131:0/1448335104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f380047d0 0x7f3f38004c50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c001680 tx=0x7f3f3c01c760 comp rx=0 tx=0).stop 2023-09-12T14:55:52.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.164+0000 7f3f5348f640 1 -- 172.21.15.131:0/1448335104 shutdown_connections 2023-09-12T14:55:52.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.164+0000 7f3f5348f640 1 --2- 172.21.15.131:0/1448335104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f380047d0 0x7f3f38004c50 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:52.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.164+0000 7f3f5348f640 1 -- 172.21.15.131:0/1448335104 >> 172.21.15.131:0/1448335104 conn(0x7f3f4c069840 msgr2=0x7f3f4c0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:52.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.164+0000 7f3f5348f640 1 -- 172.21.15.131:0/1448335104 shutdown_connections 2023-09-12T14:55:52.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.164+0000 7f3f5348f640 1 -- 172.21.15.131:0/1448335104 wait complete. 2023-09-12T14:55:52.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f5348f640 1 Processor -- start 2023-09-12T14:55:52.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f5348f640 1 -- start start 2023-09-12T14:55:52.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f5348f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f380047d0 0x7f3f4c15a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:52.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f5348f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f4c1f2430 0x7f3f4c14fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:52.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f5348f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f4c150430 0x7f3f4c1573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:52.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f5348f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c1508b0 con 0x7f3f380047d0 2023-09-12T14:55:52.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f5348f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c150ab0 con 0x7f3f4c150430 2023-09-12T14:55:52.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f5348f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c1595e0 con 0x7f3f4c1f2430 2023-09-12T14:55:52.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f5122b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f380047d0 0x7f3f4c15a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:52.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f5122b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f380047d0 0x7f3f4c15a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51020/0 (socket says 172.21.15.131:51020) 2023-09-12T14:55:52.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f5122b640 1 -- 172.21.15.131:0/3516463654 learned_addr learned my addr 172.21.15.131:0/3516463654 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:52.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f50a2a640 1 --2- 172.21.15.131:0/3516463654 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f4c1f2430 0x7f3f4c14fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:52.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f51a2c640 1 --2- 172.21.15.131:0/3516463654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f4c150430 0x7f3f4c1573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:52.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f5122b640 1 -- 172.21.15.131:0/3516463654 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f4c1f2430 msgr2=0x7f3f4c14fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:52.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f5122b640 1 --2- 172.21.15.131:0/3516463654 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f4c1f2430 0x7f3f4c14fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:52.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f5122b640 1 -- 172.21.15.131:0/3516463654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f4c150430 msgr2=0x7f3f4c1573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:52.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f5122b640 1 --2- 172.21.15.131:0/3516463654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f4c150430 0x7f3f4c1573c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:52.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f5122b640 1 -- 172.21.15.131:0/3516463654 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c1598c0 con 0x7f3f380047d0 2023-09-12T14:55:52.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f5122b640 1 --2- 172.21.15.131:0/3516463654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f380047d0 0x7f3f4c15a3a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3f48011a50 tx=0x7f3f48011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:52.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f41ffb640 1 -- 172.21.15.131:0/3516463654 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4800fd90 con 0x7f3f380047d0 2023-09-12T14:55:52.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f41ffb640 1 -- 172.21.15.131:0/3516463654 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3f48012450 con 0x7f3f380047d0 2023-09-12T14:55:52.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f5348f640 1 -- 172.21.15.131:0/3516463654 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f4c159ba0 con 0x7f3f380047d0 2023-09-12T14:55:52.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f41ffb640 1 -- 172.21.15.131:0/3516463654 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48012800 con 0x7f3f380047d0 2023-09-12T14:55:52.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f5348f640 1 -- 172.21.15.131:0/3516463654 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f4c1f2890 con 0x7f3f380047d0 2023-09-12T14:55:52.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f5348f640 1 -- 172.21.15.131:0/3516463654 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f4c073580 con 0x7f3f380047d0 2023-09-12T14:55:52.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f41ffb640 1 -- 172.21.15.131:0/3516463654 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f3f48012960 con 0x7f3f380047d0 2023-09-12T14:55:52.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f41ffb640 1 --2- 172.21.15.131:0/3516463654 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3f30065b20 0x7f3f30067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:52.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f41ffb640 1 -- 172.21.15.131:0/3516463654 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3f48083e10 con 0x7f3f380047d0 2023-09-12T14:55:52.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.168+0000 7f3f50a2a640 1 --2- 172.21.15.131:0/3516463654 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3f30065b20 0x7f3f30067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:52.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.172+0000 7f3f50a2a640 1 --2- 172.21.15.131:0/3516463654 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3f30065b20 0x7f3f30067fe0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c0abee0 tx=0x7f3f3c019500 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:52.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.176+0000 7f3f41ffb640 1 -- 172.21.15.131:0/3516463654 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f3f48051bf0 con 0x7f3f380047d0 2023-09-12T14:55:52.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.388+0000 7f3f5348f640 1 -- 172.21.15.131:0/3516463654 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f3f4c0cd4d0 con 0x7f3f380047d0 2023-09-12T14:55:52.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.396+0000 7f3f41ffb640 1 -- 172.21.15.131:0/3516463654 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v16) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3f4801f020 con 0x7f3f380047d0 2023-09-12T14:55:52.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.396+0000 7f3f5348f640 1 -- 172.21.15.131:0/3516463654 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3f30065b20 msgr2=0x7f3f30067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:52.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.396+0000 7f3f5348f640 1 --2- 172.21.15.131:0/3516463654 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3f30065b20 0x7f3f30067fe0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c0abee0 tx=0x7f3f3c019500 comp rx=0 tx=0).stop 2023-09-12T14:55:52.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.396+0000 7f3f5348f640 1 -- 172.21.15.131:0/3516463654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f380047d0 msgr2=0x7f3f4c15a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:52.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.396+0000 7f3f5348f640 1 --2- 172.21.15.131:0/3516463654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f380047d0 0x7f3f4c15a3a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3f48011a50 tx=0x7f3f48011f10 comp rx=0 tx=0).stop 2023-09-12T14:55:52.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.396+0000 7f3f5348f640 1 -- 172.21.15.131:0/3516463654 shutdown_connections 2023-09-12T14:55:52.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.396+0000 7f3f5348f640 1 --2- 172.21.15.131:0/3516463654 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f3f30065b20 0x7f3f30067fe0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:52.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.396+0000 7f3f5348f640 1 --2- 172.21.15.131:0/3516463654 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f4c1f2430 0x7f3f4c14fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:52.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.396+0000 7f3f5348f640 1 --2- 172.21.15.131:0/3516463654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f4c150430 0x7f3f4c1573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:52.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.396+0000 7f3f5348f640 1 --2- 172.21.15.131:0/3516463654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f380047d0 0x7f3f4c15a3a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:52.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.396+0000 7f3f5348f640 1 -- 172.21.15.131:0/3516463654 >> 172.21.15.131:0/3516463654 conn(0x7f3f4c069840 msgr2=0x7f3f4c0adde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:52.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.396+0000 7f3f5348f640 1 -- 172.21.15.131:0/3516463654 shutdown_connections 2023-09-12T14:55:52.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.396+0000 7f3f5348f640 1 -- 172.21.15.131:0/3516463654 wait complete. 2023-09-12T14:55:52.418 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-09-12T14:55:52.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.540+0000 7f43c131c640 1 -- 172.21.15.131:0/2196850323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43bc0cfab0 msgr2=0x7f43bc0ccc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:52.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.540+0000 7f43c131c640 1 --2- 172.21.15.131:0/2196850323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43bc0cfab0 0x7f43bc0ccc00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f43bc0c2860 tx=0x7f43a401c7c0 comp rx=0 tx=0).stop 2023-09-12T14:55:52.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.540+0000 7f43c131c640 1 -- 172.21.15.131:0/2196850323 shutdown_connections 2023-09-12T14:55:52.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.540+0000 7f43c131c640 1 --2- 172.21.15.131:0/2196850323 >> v2:172.21.15.187:3300/0 conn(0x7f43bc15c6d0 0x7f43bc0ace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:52.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.540+0000 7f43c131c640 1 --2- 172.21.15.131:0/2196850323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43bc0cfab0 0x7f43bc0ccc00 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:52.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.540+0000 7f43c131c640 1 -- 172.21.15.131:0/2196850323 >> 172.21.15.131:0/2196850323 conn(0x7f43bc098040 msgr2=0x7f43bc0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:52.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.540+0000 7f43c131c640 1 -- 172.21.15.131:0/2196850323 shutdown_connections 2023-09-12T14:55:52.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.540+0000 7f43c131c640 1 -- 172.21.15.131:0/2196850323 wait complete. 2023-09-12T14:55:52.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.540+0000 7f43c131c640 1 Processor -- start 2023-09-12T14:55:52.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.540+0000 7f43c131c640 1 -- start start 2023-09-12T14:55:52.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43c131c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43bc0cfab0 0x7f43bc14eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:52.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43c131c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43bc15c2f0 0x7f43bc154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:52.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43c131c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43bc15c6d0 0x7f43bc157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:52.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43c131c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc159940 con 0x7f43bc15c6d0 2023-09-12T14:55:52.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43c131c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc15a4c0 con 0x7f43bc15c2f0 2023-09-12T14:55:52.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43c131c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc14f430 con 0x7f43bc0cfab0 2023-09-12T14:55:52.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43bb7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43bc15c6d0 0x7f43bc157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:52.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43bb7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43bc15c6d0 0x7f43bc157410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51036/0 (socket says 172.21.15.131:51036) 2023-09-12T14:55:52.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43bb7fe640 1 -- 172.21.15.131:0/2716715555 learned_addr learned my addr 172.21.15.131:0/2716715555 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:52.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43baffd640 1 --2- 172.21.15.131:0/2716715555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43bc0cfab0 0x7f43bc14eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:52.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43bb7fe640 1 -- 172.21.15.131:0/2716715555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43bc0cfab0 msgr2=0x7f43bc14eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:52.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43ba7fc640 1 --2- 172.21.15.131:0/2716715555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43bc15c2f0 0x7f43bc154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:52.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43bb7fe640 1 --2- 172.21.15.131:0/2716715555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43bc0cfab0 0x7f43bc14eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:52.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43bb7fe640 1 -- 172.21.15.131:0/2716715555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43bc15c2f0 msgr2=0x7f43bc154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:52.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43bb7fe640 1 --2- 172.21.15.131:0/2716715555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43bc15c2f0 0x7f43bc154910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:52.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43bb7fe640 1 -- 172.21.15.131:0/2716715555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a40094a0 con 0x7f43bc15c6d0 2023-09-12T14:55:52.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43bb7fe640 1 --2- 172.21.15.131:0/2716715555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43bc15c6d0 0x7f43bc157410 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f43ac00d5e0 tx=0x7f43ac00daa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:52.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43b37fe640 1 -- 172.21.15.131:0/2716715555 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ac005220 con 0x7f43bc15c6d0 2023-09-12T14:55:52.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43b37fe640 1 -- 172.21.15.131:0/2716715555 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f43ac005470 con 0x7f43bc15c6d0 2023-09-12T14:55:52.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43c131c640 1 -- 172.21.15.131:0/2716715555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43bc159b90 con 0x7f43bc15c6d0 2023-09-12T14:55:52.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43b37fe640 1 -- 172.21.15.131:0/2716715555 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ac0047b0 con 0x7f43bc15c6d0 2023-09-12T14:55:52.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43c131c640 1 -- 172.21.15.131:0/2716715555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43bc15a150 con 0x7f43bc15c6d0 2023-09-12T14:55:52.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43c131c640 1 -- 172.21.15.131:0/2716715555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43bc15a630 con 0x7f43bc15c6d0 2023-09-12T14:55:52.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43b37fe640 1 -- 172.21.15.131:0/2716715555 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f43ac004930 con 0x7f43bc15c6d0 2023-09-12T14:55:52.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43b37fe640 1 --2- 172.21.15.131:0/2716715555 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f439c065b50 0x7f439c068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:52.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43baffd640 1 --2- 172.21.15.131:0/2716715555 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f439c065b50 0x7f439c068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:52.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43b37fe640 1 -- 172.21.15.131:0/2716715555 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f43ac082f50 con 0x7f43bc15c6d0 2023-09-12T14:55:52.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.544+0000 7f43baffd640 1 --2- 172.21.15.131:0/2716715555 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f439c065b50 0x7f439c068010 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f43b4001600 tx=0x7f43b4009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:52.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.548+0000 7f43b37fe640 1 -- 172.21.15.131:0/2716715555 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f43ac050c80 con 0x7f43bc15c6d0 2023-09-12T14:55:52.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.756+0000 7f43c131c640 1 -- 172.21.15.131:0/2716715555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f43bc0adf70 con 0x7f43bc15c6d0 2023-09-12T14:55:52.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.764+0000 7f43b37fe640 1 -- 172.21.15.131:0/2716715555 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v17) v1 ==== 90+0+0 (secure 0 0 0) 0x7f43ac017020 con 0x7f43bc15c6d0 2023-09-12T14:55:52.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.764+0000 7f43c131c640 1 -- 172.21.15.131:0/2716715555 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f439c065b50 msgr2=0x7f439c068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:52.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.764+0000 7f43c131c640 1 --2- 172.21.15.131:0/2716715555 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f439c065b50 0x7f439c068010 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f43b4001600 tx=0x7f43b4009040 comp rx=0 tx=0).stop 2023-09-12T14:55:52.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.764+0000 7f43c131c640 1 -- 172.21.15.131:0/2716715555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43bc15c6d0 msgr2=0x7f43bc157410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:52.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.764+0000 7f43c131c640 1 --2- 172.21.15.131:0/2716715555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43bc15c6d0 0x7f43bc157410 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f43ac00d5e0 tx=0x7f43ac00daa0 comp rx=0 tx=0).stop 2023-09-12T14:55:52.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.764+0000 7f43c131c640 1 -- 172.21.15.131:0/2716715555 shutdown_connections 2023-09-12T14:55:52.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.764+0000 7f43c131c640 1 --2- 172.21.15.131:0/2716715555 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f439c065b50 0x7f439c068010 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:52.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.764+0000 7f43c131c640 1 --2- 172.21.15.131:0/2716715555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43bc15c6d0 0x7f43bc157410 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:52.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.764+0000 7f43c131c640 1 --2- 172.21.15.131:0/2716715555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43bc15c2f0 0x7f43bc154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:52.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.764+0000 7f43c131c640 1 --2- 172.21.15.131:0/2716715555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43bc0cfab0 0x7f43bc14eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:52.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.764+0000 7f43c131c640 1 -- 172.21.15.131:0/2716715555 >> 172.21.15.131:0/2716715555 conn(0x7f43bc098040 msgr2=0x7f43bc06daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:52.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.764+0000 7f43c131c640 1 -- 172.21.15.131:0/2716715555 shutdown_connections 2023-09-12T14:55:52.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.764+0000 7f43c131c640 1 -- 172.21.15.131:0/2716715555 wait complete. 2023-09-12T14:55:52.784 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-09-12T14:55:52.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f83883f1640 1 -- 172.21.15.131:0/2758236708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8380158d70 msgr2=0x7f838006b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:52.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f83883f1640 1 --2- 172.21.15.131:0/2758236708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8380158d70 0x7f838006b430 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8380070ff0 tx=0x7f837c01c7d0 comp rx=0 tx=0).stop 2023-09-12T14:55:52.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f83883f1640 1 -- 172.21.15.131:0/2758236708 shutdown_connections 2023-09-12T14:55:52.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f83883f1640 1 --2- 172.21.15.131:0/2758236708 >> v2:172.21.15.187:3300/0 conn(0x7f83800abae0 0x7f83800c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:52.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f83883f1640 1 --2- 172.21.15.131:0/2758236708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8380158d70 0x7f838006b430 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:52.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f83883f1640 1 -- 172.21.15.131:0/2758236708 >> 172.21.15.131:0/2758236708 conn(0x7f8380069840 msgr2=0x7f83800cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:52.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f83883f1640 1 -- 172.21.15.131:0/2758236708 shutdown_connections 2023-09-12T14:55:52.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f83883f1640 1 -- 172.21.15.131:0/2758236708 wait complete. 2023-09-12T14:55:52.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f83883f1640 1 Processor -- start 2023-09-12T14:55:52.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f83883f1640 1 -- start start 2023-09-12T14:55:52.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f83883f1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83800abae0 0x7f8380070960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:52.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f83883f1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83800cc430 0x7f838006f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:52.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f83883f1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8380158d70 0x7f838014fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:52.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f83883f1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8380157c00 con 0x7f83800cc430 2023-09-12T14:55:52.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f83883f1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838006fe00 con 0x7f8380158d70 2023-09-12T14:55:52.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f83883f1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83801573a0 con 0x7f83800abae0 2023-09-12T14:55:52.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f838598c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83800cc430 0x7f838006f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:52.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f838598c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83800cc430 0x7f838006f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51052/0 (socket says 172.21.15.131:51052) 2023-09-12T14:55:52.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f838598c640 1 -- 172.21.15.131:0/1943206103 learned_addr learned my addr 172.21.15.131:0/1943206103 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:52.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f838618d640 1 --2- 172.21.15.131:0/1943206103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83800abae0 0x7f8380070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:52.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f838698e640 1 --2- 172.21.15.131:0/1943206103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8380158d70 0x7f838014fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:52.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f838598c640 1 -- 172.21.15.131:0/1943206103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83800abae0 msgr2=0x7f8380070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:52.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f838598c640 1 --2- 172.21.15.131:0/1943206103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83800abae0 0x7f8380070960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:52.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f838598c640 1 -- 172.21.15.131:0/1943206103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8380158d70 msgr2=0x7f838014fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:52.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f838598c640 1 --2- 172.21.15.131:0/1943206103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8380158d70 0x7f838014fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:52.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f838598c640 1 -- 172.21.15.131:0/1943206103 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f837c009560 con 0x7f83800cc430 2023-09-12T14:55:52.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f838698e640 1 --2- 172.21.15.131:0/1943206103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8380158d70 0x7f838014fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:55:52.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f838598c640 1 --2- 172.21.15.131:0/1943206103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83800cc430 0x7f838006f8c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8370005d10 tx=0x7f837000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:52.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f8376ffd640 1 -- 172.21.15.131:0/1943206103 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8370009c30 con 0x7f83800cc430 2023-09-12T14:55:52.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f8376ffd640 1 -- 172.21.15.131:0/1943206103 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8370009e80 con 0x7f83800cc430 2023-09-12T14:55:52.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f83883f1640 1 -- 172.21.15.131:0/1943206103 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8380157640 con 0x7f83800cc430 2023-09-12T14:55:52.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f8376ffd640 1 -- 172.21.15.131:0/1943206103 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8370005280 con 0x7f83800cc430 2023-09-12T14:55:52.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.912+0000 7f83883f1640 1 -- 172.21.15.131:0/1943206103 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8380070010 con 0x7f83800cc430 2023-09-12T14:55:52.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.916+0000 7f83883f1640 1 -- 172.21.15.131:0/1943206103 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8380073580 con 0x7f83800cc430 2023-09-12T14:55:52.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.916+0000 7f8376ffd640 1 -- 172.21.15.131:0/1943206103 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f83700053e0 con 0x7f83800cc430 2023-09-12T14:55:52.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.916+0000 7f8376ffd640 1 --2- 172.21.15.131:0/1943206103 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8364065b20 0x7f8364067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:52.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.916+0000 7f8376ffd640 1 -- 172.21.15.131:0/1943206103 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f83700831c0 con 0x7f83800cc430 2023-09-12T14:55:52.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.916+0000 7f838618d640 1 --2- 172.21.15.131:0/1943206103 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8364065b20 0x7f8364067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:52.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.916+0000 7f838618d640 1 --2- 172.21.15.131:0/1943206103 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8364065b20 0x7f8364067fe0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f837c0ae000 tx=0x7f837c01b010 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:52.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:52.920+0000 7f8376ffd640 1 -- 172.21.15.131:0/1943206103 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f8370050e80 con 0x7f83800cc430 2023-09-12T14:55:53.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.116+0000 7f83883f1640 1 -- 172.21.15.131:0/1943206103 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f83800cd4d0 con 0x7f83800cc430 2023-09-12T14:55:53.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.120+0000 7f8376ffd640 1 -- 172.21.15.131:0/1943206103 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v18) v1 ==== 90+0+0 (secure 0 0 0) 0x7f83700544a0 con 0x7f83800cc430 2023-09-12T14:55:53.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.124+0000 7f83883f1640 1 -- 172.21.15.131:0/1943206103 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8364065b20 msgr2=0x7f8364067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:53.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.124+0000 7f83883f1640 1 --2- 172.21.15.131:0/1943206103 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8364065b20 0x7f8364067fe0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f837c0ae000 tx=0x7f837c01b010 comp rx=0 tx=0).stop 2023-09-12T14:55:53.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.124+0000 7f83883f1640 1 -- 172.21.15.131:0/1943206103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83800cc430 msgr2=0x7f838006f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:53.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.124+0000 7f83883f1640 1 --2- 172.21.15.131:0/1943206103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83800cc430 0x7f838006f8c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8370005d10 tx=0x7f837000dde0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.124+0000 7f83883f1640 1 -- 172.21.15.131:0/1943206103 shutdown_connections 2023-09-12T14:55:53.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.124+0000 7f83883f1640 1 --2- 172.21.15.131:0/1943206103 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8364065b20 0x7f8364067fe0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.124+0000 7f83883f1640 1 --2- 172.21.15.131:0/1943206103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8380158d70 0x7f838014fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.124+0000 7f83883f1640 1 --2- 172.21.15.131:0/1943206103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83800cc430 0x7f838006f8c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.124+0000 7f83883f1640 1 --2- 172.21.15.131:0/1943206103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83800abae0 0x7f8380070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.124+0000 7f83883f1640 1 -- 172.21.15.131:0/1943206103 >> 172.21.15.131:0/1943206103 conn(0x7f8380069840 msgr2=0x7f83800ae690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:53.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.124+0000 7f83883f1640 1 -- 172.21.15.131:0/1943206103 shutdown_connections 2023-09-12T14:55:53.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.124+0000 7f83883f1640 1 -- 172.21.15.131:0/1943206103 wait complete. 2023-09-12T14:55:53.142 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-09-12T14:55:53.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.264+0000 7fec20cf7640 1 -- 172.21.15.131:0/1665347555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1c0d1680 msgr2=0x7fec1c0b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:53.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.264+0000 7fec20cf7640 1 --2- 172.21.15.131:0/1665347555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1c0d1680 0x7fec1c0b0750 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fec1c08b610 tx=0x7fec0c01c830 comp rx=0 tx=0).stop 2023-09-12T14:55:53.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.264+0000 7fec20cf7640 1 -- 172.21.15.131:0/1665347555 shutdown_connections 2023-09-12T14:55:53.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.264+0000 7fec20cf7640 1 --2- 172.21.15.131:0/1665347555 >> v2:172.21.15.187:3300/0 conn(0x7fec1c15e9f0 0x7fec1c0b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.264+0000 7fec20cf7640 1 --2- 172.21.15.131:0/1665347555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1c0d1680 0x7fec1c0b0750 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.264+0000 7fec20cf7640 1 -- 172.21.15.131:0/1665347555 >> 172.21.15.131:0/1665347555 conn(0x7fec1c09a200 msgr2=0x7fec1c0d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:53.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.264+0000 7fec20cf7640 1 -- 172.21.15.131:0/1665347555 shutdown_connections 2023-09-12T14:55:53.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.264+0000 7fec20cf7640 1 -- 172.21.15.131:0/1665347555 wait complete. 2023-09-12T14:55:53.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.264+0000 7fec20cf7640 1 Processor -- start 2023-09-12T14:55:53.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.264+0000 7fec20cf7640 1 -- start start 2023-09-12T14:55:53.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.264+0000 7fec20cf7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec1c0cf2d0 0x7fec1c0b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:53.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.264+0000 7fec20cf7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1c0d1680 0x7fec1c14d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:53.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.264+0000 7fec20cf7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec1c15e9f0 0x7fec1c0b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:53.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.264+0000 7fec20cf7640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c0b4620 con 0x7fec1c0d1680 2023-09-12T14:55:53.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.264+0000 7fec20cf7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c0b4900 con 0x7fec1c15e9f0 2023-09-12T14:55:53.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.264+0000 7fec20cf7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c0b4be0 con 0x7fec1c0cf2d0 2023-09-12T14:55:53.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.264+0000 7fec19d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1c0d1680 0x7fec1c14d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:53.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.264+0000 7fec19d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1c0d1680 0x7fec1c14d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51076/0 (socket says 172.21.15.131:51076) 2023-09-12T14:55:53.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.264+0000 7fec19d9b640 1 -- 172.21.15.131:0/706208342 learned_addr learned my addr 172.21.15.131:0/706208342 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:53.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.264+0000 7fec1a59c640 1 --2- 172.21.15.131:0/706208342 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec1c0cf2d0 0x7fec1c0b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:53.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.264+0000 7fec1ad9d640 1 --2- 172.21.15.131:0/706208342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec1c15e9f0 0x7fec1c0b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:53.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.264+0000 7fec19d9b640 1 -- 172.21.15.131:0/706208342 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec1c0cf2d0 msgr2=0x7fec1c0b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:53.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.268+0000 7fec19d9b640 1 --2- 172.21.15.131:0/706208342 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec1c0cf2d0 0x7fec1c0b56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.268+0000 7fec19d9b640 1 -- 172.21.15.131:0/706208342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec1c15e9f0 msgr2=0x7fec1c0b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:53.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.268+0000 7fec19d9b640 1 --2- 172.21.15.131:0/706208342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec1c15e9f0 0x7fec1c0b3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.268+0000 7fec19d9b640 1 -- 172.21.15.131:0/706208342 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0c009560 con 0x7fec1c0d1680 2023-09-12T14:55:53.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.268+0000 7fec1ad9d640 1 --2- 172.21.15.131:0/706208342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec1c15e9f0 0x7fec1c0b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:53.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.268+0000 7fec19d9b640 1 --2- 172.21.15.131:0/706208342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1c0d1680 0x7fec1c14d7e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0b0040 tx=0x7fec0c0a58d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:53.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.268+0000 7fec0affd640 1 -- 172.21.15.131:0/706208342 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c005230 con 0x7fec1c0d1680 2023-09-12T14:55:53.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.268+0000 7fec0affd640 1 -- 172.21.15.131:0/706208342 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fec0c0054a0 con 0x7fec1c0d1680 2023-09-12T14:55:53.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.268+0000 7fec20cf7640 1 -- 172.21.15.131:0/706208342 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec1c0b4ec0 con 0x7fec1c0d1680 2023-09-12T14:55:53.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.268+0000 7fec0affd640 1 -- 172.21.15.131:0/706208342 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c004930 con 0x7fec1c0d1680 2023-09-12T14:55:53.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.268+0000 7fec20cf7640 1 -- 172.21.15.131:0/706208342 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec1c14dd20 con 0x7fec1c0d1680 2023-09-12T14:55:53.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.268+0000 7fec20cf7640 1 -- 172.21.15.131:0/706208342 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec1c073580 con 0x7fec1c0d1680 2023-09-12T14:55:53.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.268+0000 7fec0affd640 1 -- 172.21.15.131:0/706208342 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fec0c0b4070 con 0x7fec1c0d1680 2023-09-12T14:55:53.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.268+0000 7fec0affd640 1 --2- 172.21.15.131:0/706208342 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7febfc065b20 0x7febfc067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:53.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.268+0000 7fec0affd640 1 -- 172.21.15.131:0/706208342 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fec0c11c5e0 con 0x7fec1c0d1680 2023-09-12T14:55:53.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.268+0000 7fec1a59c640 1 --2- 172.21.15.131:0/706208342 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7febfc065b20 0x7febfc067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:53.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.268+0000 7fec1a59c640 1 --2- 172.21.15.131:0/706208342 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7febfc065b20 0x7febfc067fe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0c6f20 tx=0x7fec10009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:53.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.272+0000 7fec0affd640 1 -- 172.21.15.131:0/706208342 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fec0c0ea290 con 0x7fec1c0d1680 2023-09-12T14:55:53.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.480+0000 7fec20cf7640 1 -- 172.21.15.131:0/706208342 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fec1c0b0020 con 0x7fec1c0d1680 2023-09-12T14:55:53.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.484+0000 7fec0affd640 1 -- 172.21.15.131:0/706208342 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v19) v1 ==== 97+0+0 (secure 0 0 0) 0x7fec0c0ed8b0 con 0x7fec1c0d1680 2023-09-12T14:55:53.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.484+0000 7fec20cf7640 1 -- 172.21.15.131:0/706208342 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7febfc065b20 msgr2=0x7febfc067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:53.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.484+0000 7fec20cf7640 1 --2- 172.21.15.131:0/706208342 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7febfc065b20 0x7febfc067fe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0c6f20 tx=0x7fec10009040 comp rx=0 tx=0).stop 2023-09-12T14:55:53.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.484+0000 7fec20cf7640 1 -- 172.21.15.131:0/706208342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1c0d1680 msgr2=0x7fec1c14d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:53.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.484+0000 7fec20cf7640 1 --2- 172.21.15.131:0/706208342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1c0d1680 0x7fec1c14d7e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0b0040 tx=0x7fec0c0a58d0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.484+0000 7fec20cf7640 1 -- 172.21.15.131:0/706208342 shutdown_connections 2023-09-12T14:55:53.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.484+0000 7fec20cf7640 1 --2- 172.21.15.131:0/706208342 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7febfc065b20 0x7febfc067fe0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.484+0000 7fec20cf7640 1 --2- 172.21.15.131:0/706208342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec1c15e9f0 0x7fec1c0b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.484+0000 7fec20cf7640 1 --2- 172.21.15.131:0/706208342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1c0d1680 0x7fec1c14d7e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.484+0000 7fec20cf7640 1 --2- 172.21.15.131:0/706208342 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec1c0cf2d0 0x7fec1c0b56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.484+0000 7fec20cf7640 1 -- 172.21.15.131:0/706208342 >> 172.21.15.131:0/706208342 conn(0x7fec1c09a200 msgr2=0x7fec1c0b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:53.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.484+0000 7fec20cf7640 1 -- 172.21.15.131:0/706208342 shutdown_connections 2023-09-12T14:55:53.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.484+0000 7fec20cf7640 1 -- 172.21.15.131:0/706208342 wait complete. 2023-09-12T14:55:53.504 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-09-12T14:55:53.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.624+0000 7fad869f0640 1 -- 172.21.15.131:0/1359926165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad800b2600 msgr2=0x7fad800c9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:53.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.624+0000 7fad869f0640 1 --2- 172.21.15.131:0/1359926165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad800b2600 0x7fad800c9390 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fad8006be30 tx=0x7fad7401c7b0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.624+0000 7fad869f0640 1 -- 172.21.15.131:0/1359926165 shutdown_connections 2023-09-12T14:55:53.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.624+0000 7fad869f0640 1 --2- 172.21.15.131:0/1359926165 >> v2:172.21.15.187:3300/0 conn(0x7fad800c98d0 0x7fad800af9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.624+0000 7fad869f0640 1 --2- 172.21.15.131:0/1359926165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad800b2600 0x7fad800c9390 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.624+0000 7fad869f0640 1 -- 172.21.15.131:0/1359926165 >> 172.21.15.131:0/1359926165 conn(0x7fad80069840 msgr2=0x7fad800cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:53.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.624+0000 7fad869f0640 1 -- 172.21.15.131:0/1359926165 shutdown_connections 2023-09-12T14:55:53.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.624+0000 7fad869f0640 1 -- 172.21.15.131:0/1359926165 wait complete. 2023-09-12T14:55:53.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad869f0640 1 Processor -- start 2023-09-12T14:55:53.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad869f0640 1 -- start start 2023-09-12T14:55:53.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad869f0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad800b2600 0x7fad800abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:53.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad869f0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad800c98d0 0x7fad801519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:53.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad869f0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad801574d0 0x7fad80155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:53.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad869f0640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad800ac230 con 0x7fad800b2600 2023-09-12T14:55:53.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad869f0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad800aac50 con 0x7fad800c98d0 2023-09-12T14:55:53.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad869f0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad800aaf30 con 0x7fad801574d0 2023-09-12T14:55:53.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad7ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad800b2600 0x7fad800abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:53.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad7ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad800b2600 0x7fad800abcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51098/0 (socket says 172.21.15.131:51098) 2023-09-12T14:55:53.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad7ffff640 1 -- 172.21.15.131:0/1716837918 learned_addr learned my addr 172.21.15.131:0/1716837918 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:53.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad7f7fe640 1 --2- 172.21.15.131:0/1716837918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad800c98d0 0x7fad801519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:53.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad7ffff640 1 -- 172.21.15.131:0/1716837918 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad801574d0 msgr2=0x7fad80155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:53.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad7ffff640 1 --2- 172.21.15.131:0/1716837918 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad801574d0 0x7fad80155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad7ffff640 1 -- 172.21.15.131:0/1716837918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad800c98d0 msgr2=0x7fad801519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:53.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad7ffff640 1 --2- 172.21.15.131:0/1716837918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad800c98d0 0x7fad801519b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad7ffff640 1 -- 172.21.15.131:0/1716837918 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad740094a0 con 0x7fad800b2600 2023-09-12T14:55:53.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad7ffff640 1 --2- 172.21.15.131:0/1716837918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad800b2600 0x7fad800abcf0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fad70001ec0 tx=0x7fad7000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:53.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad7cff9640 1 -- 172.21.15.131:0/1716837918 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad700054e0 con 0x7fad800b2600 2023-09-12T14:55:53.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad7cff9640 1 -- 172.21.15.131:0/1716837918 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fad70003600 con 0x7fad800b2600 2023-09-12T14:55:53.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad869f0640 1 -- 172.21.15.131:0/1716837918 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad800ab210 con 0x7fad800b2600 2023-09-12T14:55:53.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad7cff9640 1 -- 172.21.15.131:0/1716837918 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70003a10 con 0x7fad800b2600 2023-09-12T14:55:53.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad869f0640 1 -- 172.21.15.131:0/1716837918 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad800ab7d0 con 0x7fad800b2600 2023-09-12T14:55:53.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad869f0640 1 -- 172.21.15.131:0/1716837918 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad80073580 con 0x7fad800b2600 2023-09-12T14:55:53.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad7cff9640 1 -- 172.21.15.131:0/1716837918 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fad70003b70 con 0x7fad800b2600 2023-09-12T14:55:53.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad7cff9640 1 --2- 172.21.15.131:0/1716837918 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fad6c065b50 0x7fad6c068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:53.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad7cff9640 1 -- 172.21.15.131:0/1716837918 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fad70082550 con 0x7fad800b2600 2023-09-12T14:55:53.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad7f7fe640 1 --2- 172.21.15.131:0/1716837918 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fad6c065b50 0x7fad6c068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:53.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.628+0000 7fad7f7fe640 1 --2- 172.21.15.131:0/1716837918 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fad6c065b50 0x7fad6c068010 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fad740af000 tx=0x7fad7401aff0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:53.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.632+0000 7fad7cff9640 1 -- 172.21.15.131:0/1716837918 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fad700501f0 con 0x7fad800b2600 2023-09-12T14:55:53.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.824+0000 7fad869f0640 1 -- 172.21.15.131:0/1716837918 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fad800c9d20 con 0x7fad800b2600 2023-09-12T14:55:53.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.828+0000 7fad7cff9640 1 -- 172.21.15.131:0/1716837918 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v20) v1 ==== 90+0+0 (secure 0 0 0) 0x7fad800c9d20 con 0x7fad800b2600 2023-09-12T14:55:53.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.832+0000 7fad869f0640 1 -- 172.21.15.131:0/1716837918 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fad6c065b50 msgr2=0x7fad6c068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:53.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.832+0000 7fad869f0640 1 --2- 172.21.15.131:0/1716837918 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fad6c065b50 0x7fad6c068010 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fad740af000 tx=0x7fad7401aff0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.832+0000 7fad869f0640 1 -- 172.21.15.131:0/1716837918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad800b2600 msgr2=0x7fad800abcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:53.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.832+0000 7fad869f0640 1 --2- 172.21.15.131:0/1716837918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad800b2600 0x7fad800abcf0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fad70001ec0 tx=0x7fad7000c8e0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.832+0000 7fad869f0640 1 -- 172.21.15.131:0/1716837918 shutdown_connections 2023-09-12T14:55:53.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.832+0000 7fad869f0640 1 --2- 172.21.15.131:0/1716837918 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fad6c065b50 0x7fad6c068010 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.832+0000 7fad869f0640 1 --2- 172.21.15.131:0/1716837918 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad801574d0 0x7fad80155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.832+0000 7fad869f0640 1 --2- 172.21.15.131:0/1716837918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad800c98d0 0x7fad801519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.832+0000 7fad869f0640 1 --2- 172.21.15.131:0/1716837918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad800b2600 0x7fad800abcf0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.832+0000 7fad869f0640 1 -- 172.21.15.131:0/1716837918 >> 172.21.15.131:0/1716837918 conn(0x7fad80069840 msgr2=0x7fad800ad8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:53.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.832+0000 7fad869f0640 1 -- 172.21.15.131:0/1716837918 shutdown_connections 2023-09-12T14:55:53.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.832+0000 7fad869f0640 1 -- 172.21.15.131:0/1716837918 wait complete. 2023-09-12T14:55:53.851 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:55:53.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed29e0b640 1 -- 172.21.15.131:0/1593093256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed241567d0 msgr2=0x7fed24069240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:53.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed29e0b640 1 --2- 172.21.15.131:0/1593093256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed241567d0 0x7fed24069240 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fed14009840 tx=0x7fed1401c700 comp rx=0 tx=0).stop 2023-09-12T14:55:53.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed29e0b640 1 -- 172.21.15.131:0/1593093256 shutdown_connections 2023-09-12T14:55:53.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed29e0b640 1 --2- 172.21.15.131:0/1593093256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed241567d0 0x7fed24069240 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed29e0b640 1 -- 172.21.15.131:0/1593093256 >> 172.21.15.131:0/1593093256 conn(0x7fed2406b310 msgr2=0x7fed2406b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:53.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed29e0b640 1 -- 172.21.15.131:0/1593093256 shutdown_connections 2023-09-12T14:55:53.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed29e0b640 1 -- 172.21.15.131:0/1593093256 wait complete. 2023-09-12T14:55:53.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed29e0b640 1 Processor -- start 2023-09-12T14:55:53.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed29e0b640 1 -- start start 2023-09-12T14:55:53.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed29e0b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed2406d860 0x7fed24146a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:53.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed29e0b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed24071280 0x7fed2415a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:53.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed29e0b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed241567d0 0x7fed2415a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:53.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed29e0b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed24146f60 con 0x7fed2406d860 2023-09-12T14:55:53.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed29e0b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed24150760 con 0x7fed241567d0 2023-09-12T14:55:53.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed29e0b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2406aaa0 con 0x7fed24071280 2023-09-12T14:55:53.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed237fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed2406d860 0x7fed24146a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:53.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed237fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed2406d860 0x7fed24146a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51116/0 (socket says 172.21.15.131:51116) 2023-09-12T14:55:53.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed237fe640 1 -- 172.21.15.131:0/1545755695 learned_addr learned my addr 172.21.15.131:0/1545755695 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:53.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed237fe640 1 -- 172.21.15.131:0/1545755695 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed24071280 msgr2=0x7fed2415a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:53.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed237fe640 1 --2- 172.21.15.131:0/1545755695 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed24071280 0x7fed2415a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed237fe640 1 -- 172.21.15.131:0/1545755695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed241567d0 msgr2=0x7fed2415a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:53.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed237fe640 1 --2- 172.21.15.131:0/1545755695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed241567d0 0x7fed2415a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:53.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed237fe640 1 -- 172.21.15.131:0/1545755695 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed140094a0 con 0x7fed2406d860 2023-09-12T14:55:53.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed237fe640 1 --2- 172.21.15.131:0/1545755695 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed2406d860 0x7fed24146a20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fed140b0040 tx=0x7fed140a5880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:53.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed0bfff640 1 -- 172.21.15.131:0/1545755695 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed140ba070 con 0x7fed2406d860 2023-09-12T14:55:53.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed0bfff640 1 -- 172.21.15.131:0/1545755695 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fed140042b0 con 0x7fed2406d860 2023-09-12T14:55:53.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed29e0b640 1 -- 172.21.15.131:0/1545755695 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed24150bf0 con 0x7fed2406d860 2023-09-12T14:55:53.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed0bfff640 1 -- 172.21.15.131:0/1545755695 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed14003600 con 0x7fed2406d860 2023-09-12T14:55:53.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.960+0000 7fed29e0b640 1 -- 172.21.15.131:0/1545755695 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed24150110 con 0x7fed2406d860 2023-09-12T14:55:53.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.964+0000 7fed29e0b640 1 -- 172.21.15.131:0/1545755695 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed24073580 con 0x7fed2406d860 2023-09-12T14:55:53.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.964+0000 7fed0bfff640 1 -- 172.21.15.131:0/1545755695 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fed140b4040 con 0x7fed2406d860 2023-09-12T14:55:53.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.964+0000 7fed0bfff640 1 --2- 172.21.15.131:0/1545755695 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fed10065b20 0x7fed10067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:53.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.964+0000 7fed0bfff640 1 -- 172.21.15.131:0/1545755695 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fed1411d080 con 0x7fed2406d860 2023-09-12T14:55:53.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.964+0000 7fed22ffd640 1 --2- 172.21.15.131:0/1545755695 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fed10065b20 0x7fed10067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:53.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.964+0000 7fed22ffd640 1 --2- 172.21.15.131:0/1545755695 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fed10065b20 0x7fed10067fe0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fed241f5be0 tx=0x7fed180079c0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:53.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:53.968+0000 7fed0bfff640 1 -- 172.21.15.131:0/1545755695 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fed1414ddb0 con 0x7fed2406d860 2023-09-12T14:55:54.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.108+0000 7fed29e0b640 1 -- 172.21.15.131:0/1545755695 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed24152550 con 0x7fed2406d860 2023-09-12T14:55:54.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.108+0000 7fed0bfff640 1 -- 172.21.15.131:0/1545755695 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12893 (secure 0 0 0) 0x7fed140b2020 con 0x7fed2406d860 2023-09-12T14:55:54.113 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:54.114 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":29,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:55:51.247052+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:55:54.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.112+0000 7fed29e0b640 1 -- 172.21.15.131:0/1545755695 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fed10065b20 msgr2=0x7fed10067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:54.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.112+0000 7fed29e0b640 1 --2- 172.21.15.131:0/1545755695 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fed10065b20 0x7fed10067fe0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fed241f5be0 tx=0x7fed180079c0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.112+0000 7fed29e0b640 1 -- 172.21.15.131:0/1545755695 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed2406d860 msgr2=0x7fed24146a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:54.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.112+0000 7fed29e0b640 1 --2- 172.21.15.131:0/1545755695 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed2406d860 0x7fed24146a20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fed140b0040 tx=0x7fed140a5880 comp rx=0 tx=0).stop 2023-09-12T14:55:54.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.112+0000 7fed29e0b640 1 -- 172.21.15.131:0/1545755695 shutdown_connections 2023-09-12T14:55:54.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.112+0000 7fed29e0b640 1 --2- 172.21.15.131:0/1545755695 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fed10065b20 0x7fed10067fe0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.112+0000 7fed29e0b640 1 --2- 172.21.15.131:0/1545755695 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed24071280 0x7fed2415a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.112+0000 7fed29e0b640 1 --2- 172.21.15.131:0/1545755695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed241567d0 0x7fed2415a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.112+0000 7fed29e0b640 1 --2- 172.21.15.131:0/1545755695 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed2406d860 0x7fed24146a20 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.112+0000 7fed29e0b640 1 -- 172.21.15.131:0/1545755695 >> 172.21.15.131:0/1545755695 conn(0x7fed2406b310 msgr2=0x7fed2406b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:54.118 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.112+0000 7fed29e0b640 1 -- 172.21.15.131:0/1545755695 shutdown_connections 2023-09-12T14:55:54.118 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.112+0000 7fed29e0b640 1 -- 172.21.15.131:0/1545755695 wait complete. 2023-09-12T14:55:54.131 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T14:54:45.654004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-12T14:55:54.132 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T14:55:54.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f819e8d8640 1 -- 172.21.15.131:0/1956994018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81980c9370 msgr2=0x7f81980c97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:54.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f819e8d8640 1 --2- 172.21.15.131:0/1956994018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81980c9370 0x7f81980c97a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f81980cd200 tx=0x7f819001cd30 comp rx=0 tx=0).stop 2023-09-12T14:55:54.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f819e8d8640 1 -- 172.21.15.131:0/1956994018 shutdown_connections 2023-09-12T14:55:54.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f819e8d8640 1 --2- 172.21.15.131:0/1956994018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81980c9370 0x7f81980c97a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f819e8d8640 1 --2- 172.21.15.131:0/1956994018 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81980abb40 0x7f819806b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f819e8d8640 1 --2- 172.21.15.131:0/1956994018 >> v2:172.21.15.187:3300/0 conn(0x7f8198156930 0x7f8198159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f819e8d8640 1 -- 172.21.15.131:0/1956994018 >> 172.21.15.131:0/1956994018 conn(0x7f8198069840 msgr2=0x7f81980cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:54.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f819e8d8640 1 -- 172.21.15.131:0/1956994018 shutdown_connections 2023-09-12T14:55:54.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f819e8d8640 1 -- 172.21.15.131:0/1956994018 wait complete. 2023-09-12T14:55:54.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f819e8d8640 1 Processor -- start 2023-09-12T14:55:54.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f819e8d8640 1 -- start start 2023-09-12T14:55:54.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f819e8d8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81980abb40 0x7f81981508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:54.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f819e8d8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81980c9370 0x7f81980cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:54.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f819e8d8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8198156930 0x7f81981f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:54.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f819e8d8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81980ca800 con 0x7f81980c9370 2023-09-12T14:55:54.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f819e8d8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81980caa50 con 0x7f81980abb40 2023-09-12T14:55:54.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f819e8d8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81980cad30 con 0x7f8198156930 2023-09-12T14:55:54.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f818ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81980c9370 0x7f81980cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:54.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f818ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81980c9370 0x7f81980cb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51130/0 (socket says 172.21.15.131:51130) 2023-09-12T14:55:54.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f818ffff640 1 -- 172.21.15.131:0/2513764524 learned_addr learned my addr 172.21.15.131:0/2513764524 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:54.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f8197fff640 1 --2- 172.21.15.131:0/2513764524 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81980abb40 0x7f81981508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:54.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f819ce75640 1 --2- 172.21.15.131:0/2513764524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8198156930 0x7f81981f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:54.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f818ffff640 1 -- 172.21.15.131:0/2513764524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8198156930 msgr2=0x7f81981f3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:54.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f818ffff640 1 --2- 172.21.15.131:0/2513764524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8198156930 0x7f81981f3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f818ffff640 1 -- 172.21.15.131:0/2513764524 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81980abb40 msgr2=0x7f81981508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:54.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f818ffff640 1 --2- 172.21.15.131:0/2513764524 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81980abb40 0x7f81981508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f818ffff640 1 -- 172.21.15.131:0/2513764524 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81900098e0 con 0x7f81980c9370 2023-09-12T14:55:54.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f818ffff640 1 --2- 172.21.15.131:0/2513764524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81980c9370 0x7f81980cb810 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f818000c940 tx=0x7f818000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:54.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f81957fa640 1 -- 172.21.15.131:0/2513764524 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8180007bb0 con 0x7f81980c9370 2023-09-12T14:55:54.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f81957fa640 1 -- 172.21.15.131:0/2513764524 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8180007e20 con 0x7f81980c9370 2023-09-12T14:55:54.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f819e8d8640 1 -- 172.21.15.131:0/2513764524 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81980cb010 con 0x7f81980c9370 2023-09-12T14:55:54.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f81957fa640 1 -- 172.21.15.131:0/2513764524 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818000f6b0 con 0x7f81980c9370 2023-09-12T14:55:54.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.256+0000 7f819e8d8640 1 -- 172.21.15.131:0/2513764524 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8198150e30 con 0x7f81980c9370 2023-09-12T14:55:54.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.260+0000 7f819e8d8640 1 -- 172.21.15.131:0/2513764524 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8198073580 con 0x7f81980c9370 2023-09-12T14:55:54.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.260+0000 7f81957fa640 1 -- 172.21.15.131:0/2513764524 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f818000f810 con 0x7f81980c9370 2023-09-12T14:55:54.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.260+0000 7f81957fa640 1 --2- 172.21.15.131:0/2513764524 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8178065b20 0x7f8178067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:54.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.260+0000 7f81957fa640 1 -- 172.21.15.131:0/2513764524 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8180083910 con 0x7f81980c9370 2023-09-12T14:55:54.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.260+0000 7f8197fff640 1 --2- 172.21.15.131:0/2513764524 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8178065b20 0x7f8178067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:54.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.260+0000 7f8197fff640 1 --2- 172.21.15.131:0/2513764524 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8178065b20 0x7f8178067fe0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8188005fd0 tx=0x7f8188005d10 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:54.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.264+0000 7f81957fa640 1 -- 172.21.15.131:0/2513764524 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f81800515b0 con 0x7f81980c9370 2023-09-12T14:55:54.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.404+0000 7f819e8d8640 1 -- 172.21.15.131:0/2513764524 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f81980cb4c0 con 0x7f81980c9370 2023-09-12T14:55:54.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.408+0000 7f81957fa640 1 -- 172.21.15.131:0/2513764524 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8180054bd0 con 0x7f81980c9370 2023-09-12T14:55:54.410 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-09-12T14:55:54.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.408+0000 7f819e8d8640 1 -- 172.21.15.131:0/2513764524 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8178065b20 msgr2=0x7f8178067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:54.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.408+0000 7f819e8d8640 1 --2- 172.21.15.131:0/2513764524 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8178065b20 0x7f8178067fe0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8188005fd0 tx=0x7f8188005d10 comp rx=0 tx=0).stop 2023-09-12T14:55:54.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.408+0000 7f819e8d8640 1 -- 172.21.15.131:0/2513764524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81980c9370 msgr2=0x7f81980cb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:54.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.408+0000 7f819e8d8640 1 --2- 172.21.15.131:0/2513764524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81980c9370 0x7f81980cb810 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f818000c940 tx=0x7f818000ce00 comp rx=0 tx=0).stop 2023-09-12T14:55:54.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.408+0000 7f819e8d8640 1 -- 172.21.15.131:0/2513764524 shutdown_connections 2023-09-12T14:55:54.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.408+0000 7f819e8d8640 1 --2- 172.21.15.131:0/2513764524 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8178065b20 0x7f8178067fe0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.408+0000 7f819e8d8640 1 --2- 172.21.15.131:0/2513764524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8198156930 0x7f81981f3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.408+0000 7f819e8d8640 1 --2- 172.21.15.131:0/2513764524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81980c9370 0x7f81980cb810 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.408+0000 7f819e8d8640 1 --2- 172.21.15.131:0/2513764524 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81980abb40 0x7f81981508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.408+0000 7f819e8d8640 1 -- 172.21.15.131:0/2513764524 >> 172.21.15.131:0/2513764524 conn(0x7f8198069840 msgr2=0x7f8198152350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:54.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.408+0000 7f819e8d8640 1 -- 172.21.15.131:0/2513764524 shutdown_connections 2023-09-12T14:55:54.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.408+0000 7f819e8d8640 1 -- 172.21.15.131:0/2513764524 wait complete. 2023-09-12T14:55:54.427 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-12T14:55:54.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.536+0000 7fd516206640 1 -- 172.21.15.131:0/3830623088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5100b2600 msgr2=0x7fd5100c9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:54.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.536+0000 7fd516206640 1 --2- 172.21.15.131:0/3830623088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5100b2600 0x7fd5100c9390 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8009840 tx=0x7fd4f801c7d0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.536+0000 7fd516206640 1 -- 172.21.15.131:0/3830623088 shutdown_connections 2023-09-12T14:55:54.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.536+0000 7fd516206640 1 --2- 172.21.15.131:0/3830623088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5100c98d0 0x7fd5100af9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.536+0000 7fd516206640 1 --2- 172.21.15.131:0/3830623088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5100b2600 0x7fd5100c9390 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.536+0000 7fd516206640 1 -- 172.21.15.131:0/3830623088 >> 172.21.15.131:0/3830623088 conn(0x7fd510069840 msgr2=0x7fd5100cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:54.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.536+0000 7fd516206640 1 -- 172.21.15.131:0/3830623088 shutdown_connections 2023-09-12T14:55:54.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.536+0000 7fd516206640 1 -- 172.21.15.131:0/3830623088 wait complete. 2023-09-12T14:55:54.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.536+0000 7fd516206640 1 Processor -- start 2023-09-12T14:55:54.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.536+0000 7fd516206640 1 -- start start 2023-09-12T14:55:54.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd516206640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5100b2600 0x7fd510142f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:54.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd516206640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5100c98d0 0x7fd510158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:54.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd516206640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5101574d0 0x7fd5100ab930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:54.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd516206640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51014b7f0 con 0x7fd5100c98d0 2023-09-12T14:55:54.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd516206640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51014c5b0 con 0x7fd5100b2600 2023-09-12T14:55:54.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd516206640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5101434c0 con 0x7fd5101574d0 2023-09-12T14:55:54.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd50effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5100c98d0 0x7fd510158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:54.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd50effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5100c98d0 0x7fd510158240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51142/0 (socket says 172.21.15.131:51142) 2023-09-12T14:55:54.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd50effd640 1 -- 172.21.15.131:0/701659930 learned_addr learned my addr 172.21.15.131:0/701659930 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:54.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd50f7fe640 1 --2- 172.21.15.131:0/701659930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5100b2600 0x7fd510142f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:54.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd50ffff640 1 --2- 172.21.15.131:0/701659930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5101574d0 0x7fd5100ab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:54.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd50effd640 1 -- 172.21.15.131:0/701659930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5101574d0 msgr2=0x7fd5100ab930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:54.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd50effd640 1 --2- 172.21.15.131:0/701659930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5101574d0 0x7fd5100ab930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd50effd640 1 -- 172.21.15.131:0/701659930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5100b2600 msgr2=0x7fd510142f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:54.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd50effd640 1 --2- 172.21.15.131:0/701659930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5100b2600 0x7fd510142f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd50effd640 1 -- 172.21.15.131:0/701659930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f80094a0 con 0x7fd5100c98d0 2023-09-12T14:55:54.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd50effd640 1 --2- 172.21.15.131:0/701659930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5100c98d0 0x7fd510158240 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd4f801ae80 tx=0x7fd4f8005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:54.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd507fff640 1 -- 172.21.15.131:0/701659930 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f80ba070 con 0x7fd5100c98d0 2023-09-12T14:55:54.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd516206640 1 -- 172.21.15.131:0/701659930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd51014ab70 con 0x7fd5100c98d0 2023-09-12T14:55:54.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd507fff640 1 -- 172.21.15.131:0/701659930 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd4f8005020 con 0x7fd5100c98d0 2023-09-12T14:55:54.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd507fff640 1 -- 172.21.15.131:0/701659930 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8005430 con 0x7fd5100c98d0 2023-09-12T14:55:54.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd516206640 1 -- 172.21.15.131:0/701659930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd51014b130 con 0x7fd5100c98d0 2023-09-12T14:55:54.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd516206640 1 -- 172.21.15.131:0/701659930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd510073580 con 0x7fd5100c98d0 2023-09-12T14:55:54.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd507fff640 1 -- 172.21.15.131:0/701659930 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fd4f80b4040 con 0x7fd5100c98d0 2023-09-12T14:55:54.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd507fff640 1 --2- 172.21.15.131:0/701659930 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd4f0065b20 0x7fd4f0067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:54.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd507fff640 1 -- 172.21.15.131:0/701659930 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd4f811df50 con 0x7fd5100c98d0 2023-09-12T14:55:54.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd50f7fe640 1 --2- 172.21.15.131:0/701659930 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd4f0065b20 0x7fd4f0067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:54.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.540+0000 7fd50f7fe640 1 --2- 172.21.15.131:0/701659930 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd4f0065b20 0x7fd4f0067fe0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd5080046c0 tx=0x7fd508007ec0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:54.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.544+0000 7fd507fff640 1 -- 172.21.15.131:0/701659930 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fd4f80ebc80 con 0x7fd5100c98d0 2023-09-12T14:55:54.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.700+0000 7fd516206640 1 -- 172.21.15.131:0/701659930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd51014d660 con 0x7fd5100c98d0 2023-09-12T14:55:54.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.700+0000 7fd507fff640 1 -- 172.21.15.131:0/701659930 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd4f80b2070 con 0x7fd5100c98d0 2023-09-12T14:55:54.704 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:54.705 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2023-09-12T14:55:54.705 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-09-12T14:55:54.705 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2023-09-12T14:55:54.705 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2023-09-12T14:55:54.706 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:55:54.706 INFO:teuthology.orchestra.run.smithi131.stdout:] 2023-09-12T14:55:54.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.704+0000 7fd516206640 1 -- 172.21.15.131:0/701659930 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd4f0065b20 msgr2=0x7fd4f0067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:54.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.704+0000 7fd516206640 1 --2- 172.21.15.131:0/701659930 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd4f0065b20 0x7fd4f0067fe0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd5080046c0 tx=0x7fd508007ec0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.704+0000 7fd516206640 1 -- 172.21.15.131:0/701659930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5100c98d0 msgr2=0x7fd510158240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:54.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.704+0000 7fd516206640 1 --2- 172.21.15.131:0/701659930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5100c98d0 0x7fd510158240 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd4f801ae80 tx=0x7fd4f8005dd0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.704+0000 7fd516206640 1 -- 172.21.15.131:0/701659930 shutdown_connections 2023-09-12T14:55:54.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.704+0000 7fd516206640 1 --2- 172.21.15.131:0/701659930 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd4f0065b20 0x7fd4f0067fe0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.704+0000 7fd516206640 1 --2- 172.21.15.131:0/701659930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5101574d0 0x7fd5100ab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.704+0000 7fd516206640 1 --2- 172.21.15.131:0/701659930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5100c98d0 0x7fd510158240 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.704+0000 7fd516206640 1 --2- 172.21.15.131:0/701659930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5100b2600 0x7fd510142f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.704+0000 7fd516206640 1 -- 172.21.15.131:0/701659930 >> 172.21.15.131:0/701659930 conn(0x7fd510069840 msgr2=0x7fd5100c8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:54.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.704+0000 7fd516206640 1 -- 172.21.15.131:0/701659930 shutdown_connections 2023-09-12T14:55:54.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.704+0000 7fd516206640 1 -- 172.21.15.131:0/701659930 wait complete. 2023-09-12T14:55:54.723 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-12T14:55:54.724 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-12T14:55:54.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.840+0000 7f1e8ef3d640 1 -- 172.21.15.131:0/2725860555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e880c9a20 msgr2=0x7f1e8806b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:54.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.840+0000 7f1e8ef3d640 1 --2- 172.21.15.131:0/2725860555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e880c9a20 0x7f1e8806b430 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009840 tx=0x7f1e7c01c770 comp rx=0 tx=0).stop 2023-09-12T14:55:54.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.840+0000 7f1e8ef3d640 1 -- 172.21.15.131:0/2725860555 shutdown_connections 2023-09-12T14:55:54.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.840+0000 7f1e8ef3d640 1 --2- 172.21.15.131:0/2725860555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e880abae0 0x7f1e8815ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.840+0000 7f1e8ef3d640 1 --2- 172.21.15.131:0/2725860555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e880c9a20 0x7f1e8806b430 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.840+0000 7f1e8ef3d640 1 -- 172.21.15.131:0/2725860555 >> 172.21.15.131:0/2725860555 conn(0x7f1e88069840 msgr2=0x7f1e880cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:54.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.840+0000 7f1e8ef3d640 1 -- 172.21.15.131:0/2725860555 shutdown_connections 2023-09-12T14:55:54.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.840+0000 7f1e8ef3d640 1 -- 172.21.15.131:0/2725860555 wait complete. 2023-09-12T14:55:54.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e8ef3d640 1 Processor -- start 2023-09-12T14:55:54.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e8ef3d640 1 -- start start 2023-09-12T14:55:54.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e8ef3d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e880abae0 0x7f1e8814bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:54.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e8ef3d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e880c9a20 0x7f1e8813e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:54.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e8ef3d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e88158d70 0x7f1e88152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:54.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e8ef3d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8814d8d0 con 0x7f1e880c9a20 2023-09-12T14:55:54.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e8ef3d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e881506f0 con 0x7f1e88158d70 2023-09-12T14:55:54.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e8ef3d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8814c480 con 0x7f1e880abae0 2023-09-12T14:55:54.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e87fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e880c9a20 0x7f1e8813e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:54.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e87fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e880c9a20 0x7f1e8813e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51162/0 (socket says 172.21.15.131:51162) 2023-09-12T14:55:54.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e87fff640 1 -- 172.21.15.131:0/876809943 learned_addr learned my addr 172.21.15.131:0/876809943 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:54.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e8d4da640 1 --2- 172.21.15.131:0/876809943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e88158d70 0x7f1e88152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:54.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e8ccd9640 1 --2- 172.21.15.131:0/876809943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e880abae0 0x7f1e8814bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:54.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e87fff640 1 -- 172.21.15.131:0/876809943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e880abae0 msgr2=0x7f1e8814bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:54.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e87fff640 1 --2- 172.21.15.131:0/876809943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e880abae0 0x7f1e8814bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e87fff640 1 -- 172.21.15.131:0/876809943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e88158d70 msgr2=0x7f1e88152df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:54.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e87fff640 1 --2- 172.21.15.131:0/876809943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e88158d70 0x7f1e88152df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:54.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e87fff640 1 -- 172.21.15.131:0/876809943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7c0094a0 con 0x7f1e880c9a20 2023-09-12T14:55:54.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e87fff640 1 --2- 172.21.15.131:0/876809943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e880c9a20 0x7f1e8813e260 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c01d010 tx=0x7f1e7c0a6700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:54.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e857fa640 1 -- 172.21.15.131:0/876809943 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c0b6070 con 0x7f1e880c9a20 2023-09-12T14:55:54.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e857fa640 1 -- 172.21.15.131:0/876809943 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1e7c0a6d10 con 0x7f1e880c9a20 2023-09-12T14:55:54.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e8ef3d640 1 -- 172.21.15.131:0/876809943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e881548b0 con 0x7f1e880c9a20 2023-09-12T14:55:54.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e857fa640 1 -- 172.21.15.131:0/876809943 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c005280 con 0x7f1e880c9a20 2023-09-12T14:55:54.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e8ef3d640 1 -- 172.21.15.131:0/876809943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e8814dde0 con 0x7f1e880c9a20 2023-09-12T14:55:54.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e8ef3d640 1 -- 172.21.15.131:0/876809943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e8814e180 con 0x7f1e880c9a20 2023-09-12T14:55:54.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e857fa640 1 -- 172.21.15.131:0/876809943 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f1e7c0053e0 con 0x7f1e880c9a20 2023-09-12T14:55:54.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e857fa640 1 --2- 172.21.15.131:0/876809943 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1e74065b50 0x7f1e74068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:54.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e8ccd9640 1 --2- 172.21.15.131:0/876809943 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1e74065b50 0x7f1e74068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:54.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.844+0000 7f1e857fa640 1 -- 172.21.15.131:0/876809943 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1e7c11d2c0 con 0x7f1e880c9a20 2023-09-12T14:55:54.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.848+0000 7f1e8ccd9640 1 --2- 172.21.15.131:0/876809943 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1e74065b50 0x7f1e74068010 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1e78000f70 tx=0x7f1e78009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:54.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.848+0000 7f1e857fa640 1 -- 172.21.15.131:0/876809943 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f1e8814e180 con 0x7f1e880c9a20 2023-09-12T14:55:54.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:54.992+0000 7f1e8ef3d640 1 -- 172.21.15.131:0/876809943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1e8814e180 con 0x7f1e880c9a20 2023-09-12T14:55:55.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.488+0000 7f1e857fa640 1 -- 172.21.15.131:0/876809943 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1e8814e180 con 0x7f1e880c9a20 2023-09-12T14:55:55.492 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2023-09-12T14:55:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.492+0000 7f1e8ef3d640 1 -- 172.21.15.131:0/876809943 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1e74065b50 msgr2=0x7f1e74068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.492+0000 7f1e8ef3d640 1 --2- 172.21.15.131:0/876809943 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1e74065b50 0x7f1e74068010 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1e78000f70 tx=0x7f1e78009040 comp rx=0 tx=0).stop 2023-09-12T14:55:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.492+0000 7f1e8ef3d640 1 -- 172.21.15.131:0/876809943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e880c9a20 msgr2=0x7f1e8813e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.492+0000 7f1e8ef3d640 1 --2- 172.21.15.131:0/876809943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e880c9a20 0x7f1e8813e260 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c01d010 tx=0x7f1e7c0a6700 comp rx=0 tx=0).stop 2023-09-12T14:55:55.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.492+0000 7f1e8ef3d640 1 -- 172.21.15.131:0/876809943 shutdown_connections 2023-09-12T14:55:55.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.492+0000 7f1e8ef3d640 1 --2- 172.21.15.131:0/876809943 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1e74065b50 0x7f1e74068010 secure :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1e78000f70 tx=0x7f1e78009040 comp rx=0 tx=0).stop 2023-09-12T14:55:55.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.492+0000 7f1e8ef3d640 1 --2- 172.21.15.131:0/876809943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e88158d70 0x7f1e88152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:55.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.492+0000 7f1e8ef3d640 1 --2- 172.21.15.131:0/876809943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e880c9a20 0x7f1e8813e260 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:55.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.492+0000 7f1e8ef3d640 1 --2- 172.21.15.131:0/876809943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e880abae0 0x7f1e8814bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:55.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.492+0000 7f1e8ef3d640 1 -- 172.21.15.131:0/876809943 >> 172.21.15.131:0/876809943 conn(0x7f1e88069840 msgr2=0x7f1e880c8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:55.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.492+0000 7f1e8ef3d640 1 -- 172.21.15.131:0/876809943 shutdown_connections 2023-09-12T14:55:55.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.492+0000 7f1e8ef3d640 1 -- 172.21.15.131:0/876809943 wait complete. 2023-09-12T14:55:55.520 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-09-12T14:55:55.587 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-12T14:55:55.590 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-12T14:55:55.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae88ddb640 1 -- 172.21.15.131:0/4121473714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae8415e9f0 msgr2=0x7fae840b0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:55.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae88ddb640 1 --2- 172.21.15.131:0/4121473714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae8415e9f0 0x7fae840b0e60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009bc0 tx=0x7fae7c01cb50 comp rx=0 tx=0).stop 2023-09-12T14:55:55.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae88ddb640 1 -- 172.21.15.131:0/4121473714 shutdown_connections 2023-09-12T14:55:55.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae88ddb640 1 --2- 172.21.15.131:0/4121473714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae8415e9f0 0x7fae840b0e60 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:55.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae88ddb640 1 -- 172.21.15.131:0/4121473714 >> 172.21.15.131:0/4121473714 conn(0x7fae8409a200 msgr2=0x7fae840d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:55.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae88ddb640 1 -- 172.21.15.131:0/4121473714 shutdown_connections 2023-09-12T14:55:55.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae88ddb640 1 -- 172.21.15.131:0/4121473714 wait complete. 2023-09-12T14:55:55.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae88ddb640 1 Processor -- start 2023-09-12T14:55:55.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae88ddb640 1 -- start start 2023-09-12T14:55:55.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae88ddb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae840cf2d0 0x7fae8414c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:55.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae88ddb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae840d1680 0x7fae840ca860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:55.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae88ddb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae8415e9f0 0x7fae840cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:55.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae88ddb640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae840b6cb0 con 0x7fae840cf2d0 2023-09-12T14:55:55.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae88ddb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae840c8710 con 0x7fae8415e9f0 2023-09-12T14:55:55.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae88ddb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae8414ce20 con 0x7fae840d1680 2023-09-12T14:55:55.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae8259c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae840cf2d0 0x7fae8414c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:55.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae8259c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae840cf2d0 0x7fae8414c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51174/0 (socket says 172.21.15.131:51174) 2023-09-12T14:55:55.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae8259c640 1 -- 172.21.15.131:0/1064389696 learned_addr learned my addr 172.21.15.131:0/1064389696 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:55.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae82d9d640 1 --2- 172.21.15.131:0/1064389696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae8415e9f0 0x7fae840cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:55.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae7bfff640 1 --2- 172.21.15.131:0/1064389696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae840d1680 0x7fae840ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:55.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae8259c640 1 -- 172.21.15.131:0/1064389696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae840d1680 msgr2=0x7fae840ca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:55.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae8259c640 1 --2- 172.21.15.131:0/1064389696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae840d1680 0x7fae840ca860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:55.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae8259c640 1 -- 172.21.15.131:0/1064389696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae8415e9f0 msgr2=0x7fae840cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:55.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae8259c640 1 --2- 172.21.15.131:0/1064389696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae8415e9f0 0x7fae840cbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:55.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae8259c640 1 -- 172.21.15.131:0/1064389696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7c009820 con 0x7fae840cf2d0 2023-09-12T14:55:55.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae8259c640 1 --2- 172.21.15.131:0/1064389696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae840cf2d0 0x7fae8414c8e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fae7400c480 tx=0x7fae7400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:55.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae7affd640 1 -- 172.21.15.131:0/1064389696 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae74005480 con 0x7fae840cf2d0 2023-09-12T14:55:55.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae7affd640 1 -- 172.21.15.131:0/1064389696 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fae74003600 con 0x7fae840cf2d0 2023-09-12T14:55:55.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae88ddb640 1 -- 172.21.15.131:0/1064389696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae840b6410 con 0x7fae840cf2d0 2023-09-12T14:55:55.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae7affd640 1 -- 172.21.15.131:0/1064389696 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae74003a10 con 0x7fae840cf2d0 2023-09-12T14:55:55.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.652+0000 7fae88ddb640 1 -- 172.21.15.131:0/1064389696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae840b69d0 con 0x7fae840cf2d0 2023-09-12T14:55:55.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.656+0000 7fae88ddb640 1 -- 172.21.15.131:0/1064389696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae840c84e0 con 0x7fae840cf2d0 2023-09-12T14:55:55.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.656+0000 7fae7affd640 1 -- 172.21.15.131:0/1064389696 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fae74003b70 con 0x7fae840cf2d0 2023-09-12T14:55:55.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.656+0000 7fae7affd640 1 --2- 172.21.15.131:0/1064389696 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fae64065b20 0x7fae64067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:55.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.656+0000 7fae7affd640 1 -- 172.21.15.131:0/1064389696 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fae740837c0 con 0x7fae840cf2d0 2023-09-12T14:55:55.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.656+0000 7fae7bfff640 1 --2- 172.21.15.131:0/1064389696 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fae64065b20 0x7fae64067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:55.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.656+0000 7fae7bfff640 1 --2- 172.21.15.131:0/1064389696 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fae64065b20 0x7fae64067fe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fae84153a90 tx=0x7fae6c009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:55.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.660+0000 7fae7affd640 1 -- 172.21.15.131:0/1064389696 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fae740513d0 con 0x7fae840cf2d0 2023-09-12T14:55:55.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:55.812+0000 7fae88ddb640 1 -- 172.21.15.131:0/1064389696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fae840c84e0 con 0x7fae840cf2d0 2023-09-12T14:55:56.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.500+0000 7fae7affd640 1 -- 172.21.15.131:0/1064389696 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) v1 ==== 170+0+0 (secure 0 0 0) 0x7fae840c84e0 con 0x7fae840cf2d0 2023-09-12T14:55:56.505 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2023-09-12T14:55:56.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.504+0000 7fae88ddb640 1 -- 172.21.15.131:0/1064389696 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fae64065b20 msgr2=0x7fae64067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:56.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.504+0000 7fae88ddb640 1 --2- 172.21.15.131:0/1064389696 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fae64065b20 0x7fae64067fe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fae84153a90 tx=0x7fae6c009040 comp rx=0 tx=0).stop 2023-09-12T14:55:56.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.504+0000 7fae88ddb640 1 -- 172.21.15.131:0/1064389696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae840cf2d0 msgr2=0x7fae8414c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:56.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.504+0000 7fae88ddb640 1 --2- 172.21.15.131:0/1064389696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae840cf2d0 0x7fae8414c8e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fae7400c480 tx=0x7fae7400c940 comp rx=0 tx=0).stop 2023-09-12T14:55:56.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.504+0000 7fae88ddb640 1 -- 172.21.15.131:0/1064389696 shutdown_connections 2023-09-12T14:55:56.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.504+0000 7fae88ddb640 1 --2- 172.21.15.131:0/1064389696 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fae64065b20 0x7fae64067fe0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:56.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.504+0000 7fae88ddb640 1 --2- 172.21.15.131:0/1064389696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae840d1680 0x7fae840ca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:56.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.504+0000 7fae88ddb640 1 --2- 172.21.15.131:0/1064389696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae8415e9f0 0x7fae840cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:56.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.504+0000 7fae88ddb640 1 --2- 172.21.15.131:0/1064389696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae840cf2d0 0x7fae8414c8e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:56.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.504+0000 7fae88ddb640 1 -- 172.21.15.131:0/1064389696 >> 172.21.15.131:0/1064389696 conn(0x7fae8409a200 msgr2=0x7fae840cd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:56.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.504+0000 7fae88ddb640 1 -- 172.21.15.131:0/1064389696 shutdown_connections 2023-09-12T14:55:56.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.504+0000 7fae88ddb640 1 -- 172.21.15.131:0/1064389696 wait complete. 2023-09-12T14:55:56.546 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-12T14:55:56.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f792ba22640 1 -- 172.21.15.131:0/3698487403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7924158180 msgr2=0x7f79240cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:56.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f792ba22640 1 --2- 172.21.15.131:0/3698487403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7924158180 0x7f79240cae70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7914009840 tx=0x7f791401c790 comp rx=0 tx=0).stop 2023-09-12T14:55:56.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f792ba22640 1 -- 172.21.15.131:0/3698487403 shutdown_connections 2023-09-12T14:55:56.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f792ba22640 1 --2- 172.21.15.131:0/3698487403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7924158180 0x7f79240cae70 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:56.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f792ba22640 1 -- 172.21.15.131:0/3698487403 >> 172.21.15.131:0/3698487403 conn(0x7f7924069840 msgr2=0x7f79240cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:56.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f792ba22640 1 -- 172.21.15.131:0/3698487403 shutdown_connections 2023-09-12T14:55:56.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f792ba22640 1 -- 172.21.15.131:0/3698487403 wait complete. 2023-09-12T14:55:56.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f792ba22640 1 Processor -- start 2023-09-12T14:55:56.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f792ba22640 1 -- start start 2023-09-12T14:55:56.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f792ba22640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f792406b050 0x7f7924146600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:56.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f792ba22640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79240ab730 0x7f792414f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:56.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f792ba22640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7924158180 0x7f7924148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:56.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f792ba22640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7924149500 con 0x7f79240ab730 2023-09-12T14:55:56.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f792ba22640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792414f970 con 0x7f7924158180 2023-09-12T14:55:56.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f792ba22640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f792414fc50 con 0x7f792406b050 2023-09-12T14:55:56.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f7928fbd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79240ab730 0x7f792414f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:56.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f79297be640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f792406b050 0x7f7924146600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:56.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f7928fbd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79240ab730 0x7f792414f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51198/0 (socket says 172.21.15.131:51198) 2023-09-12T14:55:56.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f7928fbd640 1 -- 172.21.15.131:0/2720595619 learned_addr learned my addr 172.21.15.131:0/2720595619 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:56.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f7929fbf640 1 --2- 172.21.15.131:0/2720595619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7924158180 0x7f7924148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:56.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f7928fbd640 1 -- 172.21.15.131:0/2720595619 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f792406b050 msgr2=0x7f7924146600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:56.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f7928fbd640 1 --2- 172.21.15.131:0/2720595619 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f792406b050 0x7f7924146600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:56.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f7928fbd640 1 -- 172.21.15.131:0/2720595619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7924158180 msgr2=0x7f7924148fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:56.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f7928fbd640 1 --2- 172.21.15.131:0/2720595619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7924158180 0x7f7924148fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:56.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f7928fbd640 1 -- 172.21.15.131:0/2720595619 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79140094a0 con 0x7f79240ab730 2023-09-12T14:55:56.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f7928fbd640 1 --2- 172.21.15.131:0/2720595619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79240ab730 0x7f792414f270 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f79140b0040 tx=0x7f79140a56f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:56.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f7919ffb640 1 -- 172.21.15.131:0/2720595619 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79140ba070 con 0x7f79240ab730 2023-09-12T14:55:56.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f7919ffb640 1 -- 172.21.15.131:0/2720595619 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7914003600 con 0x7f79240ab730 2023-09-12T14:55:56.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f792ba22640 1 -- 172.21.15.131:0/2720595619 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f792414ff30 con 0x7f79240ab730 2023-09-12T14:55:56.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f7919ffb640 1 -- 172.21.15.131:0/2720595619 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79140039f0 con 0x7f79240ab730 2023-09-12T14:55:56.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.672+0000 7f792ba22640 1 -- 172.21.15.131:0/2720595619 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79241504f0 con 0x7f79240ab730 2023-09-12T14:55:56.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.676+0000 7f792ba22640 1 -- 172.21.15.131:0/2720595619 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79241506e0 con 0x7f79240ab730 2023-09-12T14:55:56.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.676+0000 7f7919ffb640 1 -- 172.21.15.131:0/2720595619 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f79140b4040 con 0x7f79240ab730 2023-09-12T14:55:56.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.676+0000 7f7919ffb640 1 --2- 172.21.15.131:0/2720595619 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7908065b20 0x7f7908067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:56.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.676+0000 7f7919ffb640 1 -- 172.21.15.131:0/2720595619 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f791411de40 con 0x7f79240ab730 2023-09-12T14:55:56.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.676+0000 7f79297be640 1 --2- 172.21.15.131:0/2720595619 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7908065b20 0x7f7908067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:56.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.676+0000 7f79297be640 1 --2- 172.21.15.131:0/2720595619 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7908065b20 0x7f7908067fe0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f79200011f0 tx=0x7f7920007d30 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:56.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.680+0000 7f7919ffb640 1 -- 172.21.15.131:0/2720595619 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f79140eba10 con 0x7f79240ab730 2023-09-12T14:55:56.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:56.864+0000 7f792ba22640 1 -- 172.21.15.131:0/2720595619 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f7924073580 con 0x7f79240ab730 2023-09-12T14:55:57.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.504+0000 7f7919ffb640 1 -- 172.21.15.131:0/2720595619 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v11) v1 ==== 172+0+0 (secure 0 0 0) 0x7f79140b2020 con 0x7f79240ab730 2023-09-12T14:55:57.508 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 4 and data pool 5 2023-09-12T14:55:57.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.508+0000 7f792ba22640 1 -- 172.21.15.131:0/2720595619 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7908065b20 msgr2=0x7f7908067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:57.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.508+0000 7f792ba22640 1 --2- 172.21.15.131:0/2720595619 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7908065b20 0x7f7908067fe0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f79200011f0 tx=0x7f7920007d30 comp rx=0 tx=0).stop 2023-09-12T14:55:57.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.508+0000 7f792ba22640 1 -- 172.21.15.131:0/2720595619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79240ab730 msgr2=0x7f792414f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:57.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.508+0000 7f792ba22640 1 --2- 172.21.15.131:0/2720595619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79240ab730 0x7f792414f270 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f79140b0040 tx=0x7f79140a56f0 comp rx=0 tx=0).stop 2023-09-12T14:55:57.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.508+0000 7f792ba22640 1 -- 172.21.15.131:0/2720595619 shutdown_connections 2023-09-12T14:55:57.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.508+0000 7f792ba22640 1 --2- 172.21.15.131:0/2720595619 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7908065b20 0x7f7908067fe0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:57.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.508+0000 7f792ba22640 1 --2- 172.21.15.131:0/2720595619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7924158180 0x7f7924148fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:57.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.508+0000 7f792ba22640 1 --2- 172.21.15.131:0/2720595619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79240ab730 0x7f792414f270 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:57.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.508+0000 7f792ba22640 1 --2- 172.21.15.131:0/2720595619 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f792406b050 0x7f7924146600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:57.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.508+0000 7f792ba22640 1 -- 172.21.15.131:0/2720595619 >> 172.21.15.131:0/2720595619 conn(0x7f7924069840 msgr2=0x7f7924072320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:57.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.508+0000 7f792ba22640 1 -- 172.21.15.131:0/2720595619 shutdown_connections 2023-09-12T14:55:57.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.508+0000 7f792ba22640 1 -- 172.21.15.131:0/2720595619 wait complete. 2023-09-12T14:55:57.543 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:55:57.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.660+0000 7f420f2d8640 1 -- 172.21.15.131:0/3428744033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41f4004d10 msgr2=0x7f41f4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:57.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.660+0000 7f420f2d8640 1 --2- 172.21.15.131:0/3428744033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41f4004d10 0x7f41f4005190 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4204009b90 tx=0x7f420401d290 comp rx=0 tx=0).stop 2023-09-12T14:55:57.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.660+0000 7f420d875640 1 -- 172.21.15.131:0/3428744033 reap_dead start 2023-09-12T14:55:57.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.660+0000 7f420f2d8640 1 -- 172.21.15.131:0/3428744033 shutdown_connections 2023-09-12T14:55:57.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.660+0000 7f420f2d8640 1 -- 172.21.15.131:0/3428744033 >> 172.21.15.131:0/3428744033 conn(0x7f42080cebe0 msgr2=0x7f42080ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:57.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.660+0000 7f420f2d8640 1 -- 172.21.15.131:0/3428744033 shutdown_connections 2023-09-12T14:55:57.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.660+0000 7f420f2d8640 1 -- 172.21.15.131:0/3428744033 wait complete. 2023-09-12T14:55:57.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.664+0000 7f420f2d8640 1 Processor -- start 2023-09-12T14:55:57.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.664+0000 7f420f2d8640 1 -- start start 2023-09-12T14:55:57.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.664+0000 7f420f2d8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41f4004d10 0x7f42081557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:57.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.664+0000 7f420f2d8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4208159a40 0x7f4208152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:57.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.664+0000 7f420f2d8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f420806b3a0 0x7f420806b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:57.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.664+0000 7f420f2d8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420806bd70 con 0x7f420806b3a0 2023-09-12T14:55:57.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.664+0000 7f420f2d8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420815afe0 con 0x7f41f4004d10 2023-09-12T14:55:57.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.664+0000 7f420f2d8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4208155d10 con 0x7f4208159a40 2023-09-12T14:55:57.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.664+0000 7f420d875640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f420806b3a0 0x7f420806b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:57.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.664+0000 7f420d875640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f420806b3a0 0x7f420806b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51214/0 (socket says 172.21.15.131:51214) 2023-09-12T14:55:57.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.664+0000 7f420d875640 1 -- 172.21.15.131:0/3151655810 learned_addr learned my addr 172.21.15.131:0/3151655810 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:57.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.664+0000 7f420c873640 1 --2- 172.21.15.131:0/3151655810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4208159a40 0x7f4208152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:57.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.664+0000 7f420d074640 1 --2- 172.21.15.131:0/3151655810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41f4004d10 0x7f42081557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:57.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.664+0000 7f420d875640 1 -- 172.21.15.131:0/3151655810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4208159a40 msgr2=0x7f4208152bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:57.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.664+0000 7f420d875640 1 --2- 172.21.15.131:0/3151655810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4208159a40 0x7f4208152bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:57.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.664+0000 7f420d875640 1 -- 172.21.15.131:0/3151655810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41f4004d10 msgr2=0x7f42081557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:57.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.664+0000 7f420d875640 1 --2- 172.21.15.131:0/3151655810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41f4004d10 0x7f42081557d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:57.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.664+0000 7f420d875640 1 -- 172.21.15.131:0/3151655810 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42081530f0 con 0x7f420806b3a0 2023-09-12T14:55:57.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.664+0000 7f420d875640 1 --2- 172.21.15.131:0/3151655810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f420806b3a0 0x7f420806b830 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4204009b90 tx=0x7f42040ad980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:57.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.664+0000 7f41f9ffb640 1 -- 172.21.15.131:0/3151655810 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42040adbf0 con 0x7f420806b3a0 2023-09-12T14:55:57.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.664+0000 7f41f9ffb640 1 -- 172.21.15.131:0/3151655810 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f42040ade40 con 0x7f420806b3a0 2023-09-12T14:55:57.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.664+0000 7f420f2d8640 1 -- 172.21.15.131:0/3151655810 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4208159f40 con 0x7f420806b3a0 2023-09-12T14:55:57.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.664+0000 7f41f9ffb640 1 -- 172.21.15.131:0/3151655810 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4204004290 con 0x7f420806b3a0 2023-09-12T14:55:57.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.664+0000 7f420f2d8640 1 -- 172.21.15.131:0/3151655810 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f420815a500 con 0x7f420806b3a0 2023-09-12T14:55:57.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.664+0000 7f420f2d8640 1 -- 172.21.15.131:0/3151655810 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42080cf060 con 0x7f420806b3a0 2023-09-12T14:55:57.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.664+0000 7f41f9ffb640 1 -- 172.21.15.131:0/3151655810 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f42040b3070 con 0x7f420806b3a0 2023-09-12T14:55:57.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.668+0000 7f41f9ffb640 1 --2- 172.21.15.131:0/3151655810 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f41ec065b20 0x7f41ec067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:57.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.668+0000 7f420d074640 1 --2- 172.21.15.131:0/3151655810 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f41ec065b20 0x7f41ec067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:57.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.668+0000 7f41f9ffb640 1 -- 172.21.15.131:0/3151655810 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f420411df80 con 0x7f420806b3a0 2023-09-12T14:55:57.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.668+0000 7f420d074640 1 --2- 172.21.15.131:0/3151655810 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f41ec065b20 0x7f41ec067fe0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4200001490 tx=0x7f420000d400 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:57.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.672+0000 7f41f9ffb640 1 -- 172.21.15.131:0/3151655810 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f42040eba10 con 0x7f420806b3a0 2023-09-12T14:55:57.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.824+0000 7f420f2d8640 1 -- 172.21.15.131:0/3151655810 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4208073580 con 0x7f420806b3a0 2023-09-12T14:55:57.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.828+0000 7f41f9ffb640 1 -- 172.21.15.131:0/3151655810 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16618 (secure 0 0 0) 0x7f42040b1030 con 0x7f420806b3a0 2023-09-12T14:55:57.830 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:57.830 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":32,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:55:56.870298+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:55:55.034593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-09-12T14:55:55.855927+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:55:57.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.828+0000 7f420f2d8640 1 -- 172.21.15.131:0/3151655810 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f41ec065b20 msgr2=0x7f41ec067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:57.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.828+0000 7f420f2d8640 1 --2- 172.21.15.131:0/3151655810 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f41ec065b20 0x7f41ec067fe0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4200001490 tx=0x7f420000d400 comp rx=0 tx=0).stop 2023-09-12T14:55:57.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.828+0000 7f420f2d8640 1 -- 172.21.15.131:0/3151655810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f420806b3a0 msgr2=0x7f420806b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:57.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.828+0000 7f420f2d8640 1 --2- 172.21.15.131:0/3151655810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f420806b3a0 0x7f420806b830 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4204009b90 tx=0x7f42040ad980 comp rx=0 tx=0).stop 2023-09-12T14:55:57.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.832+0000 7f420f2d8640 1 -- 172.21.15.131:0/3151655810 shutdown_connections 2023-09-12T14:55:57.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.832+0000 7f420f2d8640 1 --2- 172.21.15.131:0/3151655810 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f41ec065b20 0x7f41ec067fe0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:57.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.832+0000 7f420f2d8640 1 --2- 172.21.15.131:0/3151655810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4208159a40 0x7f4208152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:57.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.832+0000 7f420f2d8640 1 --2- 172.21.15.131:0/3151655810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f420806b3a0 0x7f420806b830 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:57.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.832+0000 7f420f2d8640 1 --2- 172.21.15.131:0/3151655810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41f4004d10 0x7f42081557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:57.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.832+0000 7f420f2d8640 1 -- 172.21.15.131:0/3151655810 >> 172.21.15.131:0/3151655810 conn(0x7f42080cebe0 msgr2=0x7f42080b06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:57.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.832+0000 7f420f2d8640 1 -- 172.21.15.131:0/3151655810 shutdown_connections 2023-09-12T14:55:57.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.832+0000 7f420f2d8640 1 -- 172.21.15.131:0/3151655810 wait complete. 2023-09-12T14:55:57.850 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:55:57.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f0959b8c640 1 -- 172.21.15.131:0/1832542224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f093c0047d0 msgr2=0x7f093c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:57.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f0959b8c640 1 --2- 172.21.15.131:0/1832542224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f093c0047d0 0x7f093c004c50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f09540d1ac0 tx=0x7f094801c720 comp rx=0 tx=0).stop 2023-09-12T14:55:57.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f0953fff640 1 -- 172.21.15.131:0/1832542224 reap_dead start 2023-09-12T14:55:57.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f0959b8c640 1 -- 172.21.15.131:0/1832542224 shutdown_connections 2023-09-12T14:55:57.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f0959b8c640 1 -- 172.21.15.131:0/1832542224 >> 172.21.15.131:0/1832542224 conn(0x7f095409a200 msgr2=0x7f09540d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:57.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f0959b8c640 1 -- 172.21.15.131:0/1832542224 shutdown_connections 2023-09-12T14:55:57.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f0959b8c640 1 -- 172.21.15.131:0/1832542224 wait complete. 2023-09-12T14:55:57.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f0959b8c640 1 Processor -- start 2023-09-12T14:55:57.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f0959b8c640 1 -- start start 2023-09-12T14:55:57.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f0959b8c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f093c0047d0 0x7f0954157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:57.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f0959b8c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f095415aff0 0x7f09541577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:57.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f0959b8c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09540c6860 0x7f09540c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:57.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f0959b8c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0954158870 con 0x7f09540c6860 2023-09-12T14:55:57.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f0959b8c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09540c7220 con 0x7f093c0047d0 2023-09-12T14:55:57.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f0959b8c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0954159be0 con 0x7f095415aff0 2023-09-12T14:55:57.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f0953fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09540c6860 0x7f09540c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:57.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f0953fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09540c6860 0x7f09540c6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51234/0 (socket says 172.21.15.131:51234) 2023-09-12T14:55:57.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f0953fff640 1 -- 172.21.15.131:0/2196979375 learned_addr learned my addr 172.21.15.131:0/2196979375 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:57.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f09537fe640 1 --2- 172.21.15.131:0/2196979375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f093c0047d0 0x7f0954157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:57.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f0952ffd640 1 --2- 172.21.15.131:0/2196979375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f095415aff0 0x7f09541577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:57.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f0953fff640 1 -- 172.21.15.131:0/2196979375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f095415aff0 msgr2=0x7f09541577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:57.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f0953fff640 1 --2- 172.21.15.131:0/2196979375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f095415aff0 0x7f09541577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:57.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f0953fff640 1 -- 172.21.15.131:0/2196979375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f093c0047d0 msgr2=0x7f0954157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:57.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f0953fff640 1 --2- 172.21.15.131:0/2196979375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f093c0047d0 0x7f0954157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:57.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f0953fff640 1 -- 172.21.15.131:0/2196979375 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095415b470 con 0x7f09540c6860 2023-09-12T14:55:57.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f0953fff640 1 --2- 172.21.15.131:0/2196979375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09540c6860 0x7f09540c6ce0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f095415edd0 tx=0x7f094c00fed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:57.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f093bfff640 1 -- 172.21.15.131:0/2196979375 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c00dd40 con 0x7f09540c6860 2023-09-12T14:55:57.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f093bfff640 1 -- 172.21.15.131:0/2196979375 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f094c011440 con 0x7f09540c6860 2023-09-12T14:55:57.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f0959b8c640 1 -- 172.21.15.131:0/2196979375 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0954157d30 con 0x7f09540c6860 2023-09-12T14:55:57.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f093bfff640 1 -- 172.21.15.131:0/2196979375 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c011850 con 0x7f09540c6860 2023-09-12T14:55:57.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.972+0000 7f0959b8c640 1 -- 172.21.15.131:0/2196979375 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09541592b0 con 0x7f09540c6860 2023-09-12T14:55:57.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.976+0000 7f0959b8c640 1 -- 172.21.15.131:0/2196979375 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0954073580 con 0x7f09540c6860 2023-09-12T14:55:57.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.976+0000 7f093bfff640 1 -- 172.21.15.131:0/2196979375 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f094c0119b0 con 0x7f09540c6860 2023-09-12T14:55:57.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.976+0000 7f093bfff640 1 --2- 172.21.15.131:0/2196979375 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0940065b20 0x7f0940067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:57.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.976+0000 7f093bfff640 1 -- 172.21.15.131:0/2196979375 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f094c0841f0 con 0x7f09540c6860 2023-09-12T14:55:57.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.976+0000 7f09537fe640 1 --2- 172.21.15.131:0/2196979375 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0940065b20 0x7f0940067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:57.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.976+0000 7f09537fe640 1 --2- 172.21.15.131:0/2196979375 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0940065b20 0x7f0940067fe0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f09440017b0 tx=0x7f09440011a0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:57.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:57.980+0000 7f093bfff640 1 -- 172.21.15.131:0/2196979375 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f094c051aa0 con 0x7f09540c6860 2023-09-12T14:55:58.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.136+0000 7f0959b8c640 1 -- 172.21.15.131:0/2196979375 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09540cbd50 con 0x7f09540c6860 2023-09-12T14:55:58.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.136+0000 7f093bfff640 1 -- 172.21.15.131:0/2196979375 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f094c0550c0 con 0x7f09540c6860 2023-09-12T14:55:58.141 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:58.141 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":33,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:55:57.898819+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:55:55.034593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-09-12T14:55:55.855927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:55:58.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.140+0000 7f0959b8c640 1 -- 172.21.15.131:0/2196979375 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0940065b20 msgr2=0x7f0940067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:58.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.140+0000 7f0959b8c640 1 --2- 172.21.15.131:0/2196979375 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0940065b20 0x7f0940067fe0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f09440017b0 tx=0x7f09440011a0 comp rx=0 tx=0).stop 2023-09-12T14:55:58.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.140+0000 7f0959b8c640 1 -- 172.21.15.131:0/2196979375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09540c6860 msgr2=0x7f09540c6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:58.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.140+0000 7f0959b8c640 1 --2- 172.21.15.131:0/2196979375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09540c6860 0x7f09540c6ce0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f095415edd0 tx=0x7f094c00fed0 comp rx=0 tx=0).stop 2023-09-12T14:55:58.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.140+0000 7f0959b8c640 1 -- 172.21.15.131:0/2196979375 shutdown_connections 2023-09-12T14:55:58.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.140+0000 7f0959b8c640 1 --2- 172.21.15.131:0/2196979375 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0940065b20 0x7f0940067fe0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:58.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.140+0000 7f0959b8c640 1 --2- 172.21.15.131:0/2196979375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f095415aff0 0x7f09541577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:58.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.140+0000 7f0959b8c640 1 --2- 172.21.15.131:0/2196979375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09540c6860 0x7f09540c6ce0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:58.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.140+0000 7f0959b8c640 1 --2- 172.21.15.131:0/2196979375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f093c0047d0 0x7f0954157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:58.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.140+0000 7f0959b8c640 1 -- 172.21.15.131:0/2196979375 >> 172.21.15.131:0/2196979375 conn(0x7f095409a200 msgr2=0x7f09540cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:58.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.140+0000 7f0959b8c640 1 -- 172.21.15.131:0/2196979375 shutdown_connections 2023-09-12T14:55:58.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.140+0000 7f0959b8c640 1 -- 172.21.15.131:0/2196979375 wait complete. 2023-09-12T14:55:58.160 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-12T14:55:58.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.288+0000 7fa93f1e9640 1 -- 172.21.15.131:0/2468584581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9380addb0 msgr2=0x7fa9380ac4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:58.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.288+0000 7fa93f1e9640 1 --2- 172.21.15.131:0/2468584581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9380addb0 0x7fa9380ac4b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa9380c91a0 tx=0x7fa93001ccf0 comp rx=0 tx=0).stop 2023-09-12T14:55:58.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.288+0000 7fa93f1e9640 1 -- 172.21.15.131:0/2468584581 shutdown_connections 2023-09-12T14:55:58.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.288+0000 7fa93f1e9640 1 --2- 172.21.15.131:0/2468584581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9380addb0 0x7fa9380ac4b0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:58.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.288+0000 7fa93f1e9640 1 --2- 172.21.15.131:0/2468584581 >> v2:172.21.15.187:3300/0 conn(0x7fa9380cd240 0x7fa9380abf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:58.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.288+0000 7fa93f1e9640 1 -- 172.21.15.131:0/2468584581 >> 172.21.15.131:0/2468584581 conn(0x7fa9380ceac0 msgr2=0x7fa9380ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:58.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.288+0000 7fa93f1e9640 1 -- 172.21.15.131:0/2468584581 shutdown_connections 2023-09-12T14:55:58.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.288+0000 7fa93f1e9640 1 -- 172.21.15.131:0/2468584581 wait complete. 2023-09-12T14:55:58.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.288+0000 7fa93f1e9640 1 Processor -- start 2023-09-12T14:55:58.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.288+0000 7fa93f1e9640 1 -- start start 2023-09-12T14:55:58.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.288+0000 7fa93f1e9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9380addb0 0x7fa9381548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:58.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.288+0000 7fa93f1e9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9380cd240 0x7fa9381f3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:58.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.288+0000 7fa93f1e9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa93815c3b0 0x7fa938150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:58.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.288+0000 7fa93f1e9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa938154df0 con 0x7fa9380cd240 2023-09-12T14:55:58.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.288+0000 7fa93f1e9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9381f4000 con 0x7fa9380addb0 2023-09-12T14:55:58.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.288+0000 7fa93f1e9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9381f42e0 con 0x7fa93815c3b0 2023-09-12T14:55:58.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.288+0000 7fa937fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9380cd240 0x7fa9381f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:58.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.288+0000 7fa937fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9380cd240 0x7fa9381f3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37320/0 (socket says 172.21.15.131:37320) 2023-09-12T14:55:58.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.288+0000 7fa937fff640 1 -- 172.21.15.131:0/769048836 learned_addr learned my addr 172.21.15.131:0/769048836 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:58.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.288+0000 7fa93d786640 1 --2- 172.21.15.131:0/769048836 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa93815c3b0 0x7fa938150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:58.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.288+0000 7fa93cf85640 1 --2- 172.21.15.131:0/769048836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9380addb0 0x7fa9381548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:58.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.292+0000 7fa937fff640 1 -- 172.21.15.131:0/769048836 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa93815c3b0 msgr2=0x7fa938150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:58.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.292+0000 7fa937fff640 1 --2- 172.21.15.131:0/769048836 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa93815c3b0 0x7fa938150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:58.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.292+0000 7fa937fff640 1 -- 172.21.15.131:0/769048836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9380addb0 msgr2=0x7fa9381548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:58.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.292+0000 7fa937fff640 1 --2- 172.21.15.131:0/769048836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9380addb0 0x7fa9381548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:58.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.292+0000 7fa937fff640 1 -- 172.21.15.131:0/769048836 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9300098e0 con 0x7fa9380cd240 2023-09-12T14:55:58.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.292+0000 7fa93d786640 1 --2- 172.21.15.131:0/769048836 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa93815c3b0 0x7fa938150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:55:58.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.292+0000 7fa937fff640 1 --2- 172.21.15.131:0/769048836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9380cd240 0x7fa9381f3730 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00c970 tx=0x7fa92c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:58.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.292+0000 7fa9357fa640 1 -- 172.21.15.131:0/769048836 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c007bb0 con 0x7fa9380cd240 2023-09-12T14:55:58.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.292+0000 7fa9357fa640 1 -- 172.21.15.131:0/769048836 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa92c007e20 con 0x7fa9380cd240 2023-09-12T14:55:58.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.292+0000 7fa93f1e9640 1 -- 172.21.15.131:0/769048836 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9381f45c0 con 0x7fa9380cd240 2023-09-12T14:55:58.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.292+0000 7fa9357fa640 1 -- 172.21.15.131:0/769048836 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c005280 con 0x7fa9380cd240 2023-09-12T14:55:58.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.292+0000 7fa93f1e9640 1 -- 172.21.15.131:0/769048836 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9381f4b80 con 0x7fa9380cd240 2023-09-12T14:55:58.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.292+0000 7fa93f1e9640 1 -- 172.21.15.131:0/769048836 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa93806db60 con 0x7fa9380cd240 2023-09-12T14:55:58.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.292+0000 7fa9357fa640 1 -- 172.21.15.131:0/769048836 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fa92c0053e0 con 0x7fa9380cd240 2023-09-12T14:55:58.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.292+0000 7fa9357fa640 1 --2- 172.21.15.131:0/769048836 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa924065b20 0x7fa924067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:58.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.292+0000 7fa9357fa640 1 -- 172.21.15.131:0/769048836 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa92c083be0 con 0x7fa9380cd240 2023-09-12T14:55:58.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.292+0000 7fa93cf85640 1 --2- 172.21.15.131:0/769048836 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa924065b20 0x7fa924067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:58.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.292+0000 7fa93cf85640 1 --2- 172.21.15.131:0/769048836 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa924065b20 0x7fa924067fe0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa92800b810 tx=0x7fa928008040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:58.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.296+0000 7fa9357fa640 1 -- 172.21.15.131:0/769048836 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fa93806db60 con 0x7fa9380cd240 2023-09-12T14:55:58.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.488+0000 7fa93f1e9640 1 -- 172.21.15.131:0/769048836 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa93806db60 con 0x7fa9380cd240 2023-09-12T14:55:58.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.524+0000 7fa9357fa640 1 -- 172.21.15.131:0/769048836 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v12) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa93806db60 con 0x7fa9380cd240 2023-09-12T14:55:58.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.528+0000 7fa93f1e9640 1 -- 172.21.15.131:0/769048836 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa924065b20 msgr2=0x7fa924067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:58.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.528+0000 7fa93f1e9640 1 --2- 172.21.15.131:0/769048836 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa924065b20 0x7fa924067fe0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa92800b810 tx=0x7fa928008040 comp rx=0 tx=0).stop 2023-09-12T14:55:58.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.528+0000 7fa93f1e9640 1 -- 172.21.15.131:0/769048836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9380cd240 msgr2=0x7fa9381f3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:58.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.528+0000 7fa93f1e9640 1 --2- 172.21.15.131:0/769048836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9380cd240 0x7fa9381f3730 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00c970 tx=0x7fa92c00ce30 comp rx=0 tx=0).stop 2023-09-12T14:55:58.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.528+0000 7fa93f1e9640 1 -- 172.21.15.131:0/769048836 shutdown_connections 2023-09-12T14:55:58.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.528+0000 7fa93f1e9640 1 --2- 172.21.15.131:0/769048836 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa924065b20 0x7fa924067fe0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:58.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.528+0000 7fa93f1e9640 1 --2- 172.21.15.131:0/769048836 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa93815c3b0 0x7fa938150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:58.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.528+0000 7fa93f1e9640 1 --2- 172.21.15.131:0/769048836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9380cd240 0x7fa9381f3730 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:58.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.528+0000 7fa93f1e9640 1 --2- 172.21.15.131:0/769048836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9380addb0 0x7fa9381548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:58.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.528+0000 7fa93f1e9640 1 -- 172.21.15.131:0/769048836 >> 172.21.15.131:0/769048836 conn(0x7fa9380ceac0 msgr2=0x7fa93806d110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:58.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.528+0000 7fa93f1e9640 1 -- 172.21.15.131:0/769048836 shutdown_connections 2023-09-12T14:55:58.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.528+0000 7fa93f1e9640 1 -- 172.21.15.131:0/769048836 wait complete. 2023-09-12T14:55:58.547 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-09-12T14:55:58.547 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-09-12T14:55:58.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.672+0000 7fe8b3a2e640 1 -- 172.21.15.131:0/2005157967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8ac0c9a20 msgr2=0x7fe8ac15ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:58.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.672+0000 7fe8b3a2e640 1 --2- 172.21.15.131:0/2005157967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8ac0c9a20 0x7fe8ac15ca30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac071110 tx=0x7fe8a401cad0 comp rx=0 tx=0).stop 2023-09-12T14:55:58.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.672+0000 7fe8b3a2e640 1 -- 172.21.15.131:0/2005157967 shutdown_connections 2023-09-12T14:55:58.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.672+0000 7fe8b3a2e640 1 --2- 172.21.15.131:0/2005157967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8ac0c9a20 0x7fe8ac15ca30 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:58.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.672+0000 7fe8b3a2e640 1 --2- 172.21.15.131:0/2005157967 >> v2:172.21.15.187:3300/0 conn(0x7fe8ac158d70 0x7fe8ac06b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:58.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.672+0000 7fe8b3a2e640 1 -- 172.21.15.131:0/2005157967 >> 172.21.15.131:0/2005157967 conn(0x7fe8ac069840 msgr2=0x7fe8ac0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:58.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.672+0000 7fe8b3a2e640 1 -- 172.21.15.131:0/2005157967 shutdown_connections 2023-09-12T14:55:58.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.672+0000 7fe8b3a2e640 1 -- 172.21.15.131:0/2005157967 wait complete. 2023-09-12T14:55:58.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.672+0000 7fe8b3a2e640 1 Processor -- start 2023-09-12T14:55:58.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.672+0000 7fe8b3a2e640 1 -- start start 2023-09-12T14:55:58.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.672+0000 7fe8b3a2e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8ac0abae0 0x7fe8ac14fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:58.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.672+0000 7fe8b3a2e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8ac0c9a20 0x7fe8ac0aab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:58.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.672+0000 7fe8b3a2e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8ac158d70 0x7fe8ac139e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:58.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.672+0000 7fe8b3a2e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac149610 con 0x7fe8ac0abae0 2023-09-12T14:55:58.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.672+0000 7fe8b3a2e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac147a40 con 0x7fe8ac158d70 2023-09-12T14:55:58.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.672+0000 7fe8b3a2e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac070760 con 0x7fe8ac0c9a20 2023-09-12T14:55:58.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.672+0000 7fe8b17ca640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8ac0abae0 0x7fe8ac14fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:58.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.672+0000 7fe8b17ca640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8ac0abae0 0x7fe8ac14fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37324/0 (socket says 172.21.15.131:37324) 2023-09-12T14:55:58.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.672+0000 7fe8b17ca640 1 -- 172.21.15.131:0/420396037 learned_addr learned my addr 172.21.15.131:0/420396037 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:58.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.676+0000 7fe8b1fcb640 1 --2- 172.21.15.131:0/420396037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8ac158d70 0x7fe8ac139e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:58.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.676+0000 7fe8b0fc9640 1 --2- 172.21.15.131:0/420396037 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8ac0c9a20 0x7fe8ac0aab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:58.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.676+0000 7fe8b17ca640 1 -- 172.21.15.131:0/420396037 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8ac0c9a20 msgr2=0x7fe8ac0aab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:58.696 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.676+0000 7fe8b17ca640 1 --2- 172.21.15.131:0/420396037 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8ac0c9a20 0x7fe8ac0aab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:58.696 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.676+0000 7fe8b17ca640 1 -- 172.21.15.131:0/420396037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8ac158d70 msgr2=0x7fe8ac139e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:58.696 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.676+0000 7fe8b17ca640 1 --2- 172.21.15.131:0/420396037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8ac158d70 0x7fe8ac139e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:58.696 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.676+0000 7fe8b17ca640 1 -- 172.21.15.131:0/420396037 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a4009820 con 0x7fe8ac0abae0 2023-09-12T14:55:58.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.676+0000 7fe8b0fc9640 1 --2- 172.21.15.131:0/420396037 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8ac0c9a20 0x7fe8ac0aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:58.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.676+0000 7fe8b1fcb640 1 --2- 172.21.15.131:0/420396037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8ac158d70 0x7fe8ac139e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:55:58.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.676+0000 7fe8b17ca640 1 --2- 172.21.15.131:0/420396037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8ac0abae0 0x7fe8ac14fca0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800d720 tx=0x7fe8a800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:58.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.676+0000 7fe8a1ffb640 1 -- 172.21.15.131:0/420396037 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8009c30 con 0x7fe8ac0abae0 2023-09-12T14:55:58.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.676+0000 7fe8a1ffb640 1 -- 172.21.15.131:0/420396037 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe8a8009e80 con 0x7fe8ac0abae0 2023-09-12T14:55:58.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.676+0000 7fe8b3a2e640 1 -- 172.21.15.131:0/420396037 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ac070a40 con 0x7fe8ac0abae0 2023-09-12T14:55:58.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.676+0000 7fe8a1ffb640 1 -- 172.21.15.131:0/420396037 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8005260 con 0x7fe8ac0abae0 2023-09-12T14:55:58.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.676+0000 7fe8b3a2e640 1 -- 172.21.15.131:0/420396037 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ac148e50 con 0x7fe8ac0abae0 2023-09-12T14:55:58.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.676+0000 7fe8b3a2e640 1 -- 172.21.15.131:0/420396037 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8ac149070 con 0x7fe8ac0abae0 2023-09-12T14:55:58.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.676+0000 7fe8a1ffb640 1 -- 172.21.15.131:0/420396037 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fe8a80053c0 con 0x7fe8ac0abae0 2023-09-12T14:55:58.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.676+0000 7fe8a1ffb640 1 --2- 172.21.15.131:0/420396037 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe890065b20 0x7fe890067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:58.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.676+0000 7fe8a1ffb640 1 -- 172.21.15.131:0/420396037 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8a80833f0 con 0x7fe8ac0abae0 2023-09-12T14:55:58.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.676+0000 7fe8b0fc9640 1 --2- 172.21.15.131:0/420396037 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe890065b20 0x7fe890067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:58.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.676+0000 7fe8b0fc9640 1 --2- 172.21.15.131:0/420396037 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe890065b20 0x7fe890067fe0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac1f7ef0 tx=0x7fe89c009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:58.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.680+0000 7fe8a1ffb640 1 -- 172.21.15.131:0/420396037 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fe8a8050dd0 con 0x7fe8ac0abae0 2023-09-12T14:55:58.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:58.864+0000 7fe8b3a2e640 1 -- 172.21.15.131:0/420396037 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe8ac073580 con 0x7fe8ac0abae0 2023-09-12T14:55:59.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.528+0000 7fe8a1ffb640 1 -- 172.21.15.131:0/420396037 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v13) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe8a80200c0 con 0x7fe8ac0abae0 2023-09-12T14:55:59.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.532+0000 7fe8b3a2e640 1 -- 172.21.15.131:0/420396037 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe890065b20 msgr2=0x7fe890067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:59.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.532+0000 7fe8b3a2e640 1 --2- 172.21.15.131:0/420396037 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe890065b20 0x7fe890067fe0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac1f7ef0 tx=0x7fe89c009040 comp rx=0 tx=0).stop 2023-09-12T14:55:59.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.532+0000 7fe8b3a2e640 1 -- 172.21.15.131:0/420396037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8ac0abae0 msgr2=0x7fe8ac14fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:59.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.532+0000 7fe8b3a2e640 1 --2- 172.21.15.131:0/420396037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8ac0abae0 0x7fe8ac14fca0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800d720 tx=0x7fe8a800dbe0 comp rx=0 tx=0).stop 2023-09-12T14:55:59.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.532+0000 7fe8b3a2e640 1 -- 172.21.15.131:0/420396037 shutdown_connections 2023-09-12T14:55:59.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.532+0000 7fe8b3a2e640 1 --2- 172.21.15.131:0/420396037 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe890065b20 0x7fe890067fe0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:59.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.532+0000 7fe8b3a2e640 1 --2- 172.21.15.131:0/420396037 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8ac0c9a20 0x7fe8ac0aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:59.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.532+0000 7fe8b3a2e640 1 --2- 172.21.15.131:0/420396037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8ac158d70 0x7fe8ac139e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:59.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.532+0000 7fe8b3a2e640 1 --2- 172.21.15.131:0/420396037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8ac0abae0 0x7fe8ac14fca0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:59.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.532+0000 7fe8b3a2e640 1 -- 172.21.15.131:0/420396037 >> 172.21.15.131:0/420396037 conn(0x7fe8ac069840 msgr2=0x7fe8ac0ad180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:59.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.532+0000 7fe8b3a2e640 1 -- 172.21.15.131:0/420396037 shutdown_connections 2023-09-12T14:55:59.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.532+0000 7fe8b3a2e640 1 -- 172.21.15.131:0/420396037 wait complete. 2023-09-12T14:55:59.555 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-09-12T14:55:59.556 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T14:55:59.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53fcc9e640 1 -- 172.21.15.131:0/2397717173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53f806a250 msgr2=0x7f53f80cbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:59.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53fcc9e640 1 --2- 172.21.15.131:0/2397717173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53f806a250 0x7f53f80cbb40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f53f0009840 tx=0x7f53f001c800 comp rx=0 tx=0).stop 2023-09-12T14:55:59.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53fcc9e640 1 -- 172.21.15.131:0/2397717173 shutdown_connections 2023-09-12T14:55:59.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53fcc9e640 1 --2- 172.21.15.131:0/2397717173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53f806a250 0x7f53f80cbb40 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:59.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53fcc9e640 1 --2- 172.21.15.131:0/2397717173 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53f806bb70 0x7f53f8070010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:59.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53fcc9e640 1 -- 172.21.15.131:0/2397717173 >> 172.21.15.131:0/2397717173 conn(0x7f53f80cd150 msgr2=0x7f53f80cd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:59.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53fcc9e640 1 -- 172.21.15.131:0/2397717173 shutdown_connections 2023-09-12T14:55:59.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53fcc9e640 1 -- 172.21.15.131:0/2397717173 wait complete. 2023-09-12T14:55:59.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53fcc9e640 1 Processor -- start 2023-09-12T14:55:59.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53fcc9e640 1 -- start start 2023-09-12T14:55:59.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53fcc9e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53f806a250 0x7f53f81579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:59.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53fcc9e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53f8070550 0x7f53f8156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:59.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53fcc9e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53f8157470 0x7f53f80c3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:59.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53fcc9e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f80c21f0 con 0x7f53f8070550 2023-09-12T14:55:59.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53fcc9e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f80c24d0 con 0x7f53f806a250 2023-09-12T14:55:59.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53fcc9e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f80c27b0 con 0x7f53f8157470 2023-09-12T14:55:59.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53f5d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53f8070550 0x7f53f8156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:59.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53f5d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53f8070550 0x7f53f8156cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37336/0 (socket says 172.21.15.131:37336) 2023-09-12T14:55:59.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53f5d9b640 1 -- 172.21.15.131:0/1081466960 learned_addr learned my addr 172.21.15.131:0/1081466960 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:55:59.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53f659c640 1 --2- 172.21.15.131:0/1081466960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53f806a250 0x7f53f81579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:59.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53f6d9d640 1 --2- 172.21.15.131:0/1081466960 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53f8157470 0x7f53f80c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:59.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53f5d9b640 1 -- 172.21.15.131:0/1081466960 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53f8157470 msgr2=0x7f53f80c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:59.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53f5d9b640 1 --2- 172.21.15.131:0/1081466960 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53f8157470 0x7f53f80c3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:59.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53f5d9b640 1 -- 172.21.15.131:0/1081466960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53f806a250 msgr2=0x7f53f81579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:59.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53f5d9b640 1 --2- 172.21.15.131:0/1081466960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53f806a250 0x7f53f81579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:59.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53f5d9b640 1 -- 172.21.15.131:0/1081466960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f00094a0 con 0x7f53f8070550 2023-09-12T14:55:59.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53f5d9b640 1 --2- 172.21.15.131:0/1081466960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53f8070550 0x7f53f8156cc0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f53e800cae0 tx=0x7f53e800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:59.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53eeffd640 1 -- 172.21.15.131:0/1081466960 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e8003b80 con 0x7f53f8070550 2023-09-12T14:55:59.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53eeffd640 1 -- 172.21.15.131:0/1081466960 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f53e8003df0 con 0x7f53f8070550 2023-09-12T14:55:59.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53fcc9e640 1 -- 172.21.15.131:0/1081466960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f80c2a90 con 0x7f53f8070550 2023-09-12T14:55:59.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53eeffd640 1 -- 172.21.15.131:0/1081466960 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e800aa90 con 0x7f53f8070550 2023-09-12T14:55:59.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.680+0000 7f53fcc9e640 1 -- 172.21.15.131:0/1081466960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f8157200 con 0x7f53f8070550 2023-09-12T14:55:59.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.684+0000 7f53fcc9e640 1 -- 172.21.15.131:0/1081466960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f80c2fb0 con 0x7f53f8070550 2023-09-12T14:55:59.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.684+0000 7f53eeffd640 1 -- 172.21.15.131:0/1081466960 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f53e800abf0 con 0x7f53f8070550 2023-09-12T14:55:59.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.684+0000 7f53eeffd640 1 --2- 172.21.15.131:0/1081466960 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f53d8065b20 0x7f53d8067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:55:59.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.684+0000 7f53f659c640 1 --2- 172.21.15.131:0/1081466960 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f53d8065b20 0x7f53d8067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:55:59.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.684+0000 7f53eeffd640 1 -- 172.21.15.131:0/1081466960 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53e80839b0 con 0x7f53f8070550 2023-09-12T14:55:59.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.684+0000 7f53f659c640 1 --2- 172.21.15.131:0/1081466960 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f53d8065b20 0x7f53d8067fe0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f53e0005fd0 tx=0x7f53e000a4e0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:55:59.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.688+0000 7f53eeffd640 1 -- 172.21.15.131:0/1081466960 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f53f80c2fb0 con 0x7f53f8070550 2023-09-12T14:55:59.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.872+0000 7f53fcc9e640 1 -- 172.21.15.131:0/1081466960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53f80c2fb0 con 0x7f53f8070550 2023-09-12T14:55:59.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.872+0000 7f53eeffd640 1 -- 172.21.15.131:0/1081466960 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f53f80c2fb0 con 0x7f53f8070550 2023-09-12T14:55:59.877 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:55:59.878 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4676,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/3890238785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3890238785},{"type":"v1","addr":"172.21.15.187:6834","nonce":3890238785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4978,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1436257801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":1436257801},{"type":"v1","addr":"172.21.15.187:6835","nonce":1436257801}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4990,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/806466310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":806466310},{"type":"v1","addr":"172.21.15.131:6837","nonce":806466310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-12T14:55:56.870775+0000","modified":"2023-09-12T14:55:58.869201+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/871788191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":871788191},{"type":"v1","addr":"172.21.15.131:6835","nonce":871788191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-09-12T14:55:59.878 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 13 2023-09-12T14:55:59.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.876+0000 7f53fcc9e640 1 -- 172.21.15.131:0/1081466960 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f53d8065b20 msgr2=0x7f53d8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:59.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.876+0000 7f53fcc9e640 1 --2- 172.21.15.131:0/1081466960 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f53d8065b20 0x7f53d8067fe0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f53e0005fd0 tx=0x7f53e000a4e0 comp rx=0 tx=0).stop 2023-09-12T14:55:59.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.876+0000 7f53fcc9e640 1 -- 172.21.15.131:0/1081466960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53f8070550 msgr2=0x7f53f8156cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:55:59.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.876+0000 7f53fcc9e640 1 --2- 172.21.15.131:0/1081466960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53f8070550 0x7f53f8156cc0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f53e800cae0 tx=0x7f53e800cbd0 comp rx=0 tx=0).stop 2023-09-12T14:55:59.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.876+0000 7f53fcc9e640 1 -- 172.21.15.131:0/1081466960 shutdown_connections 2023-09-12T14:55:59.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.876+0000 7f53fcc9e640 1 --2- 172.21.15.131:0/1081466960 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f53d8065b20 0x7f53d8067fe0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:59.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.876+0000 7f53fcc9e640 1 --2- 172.21.15.131:0/1081466960 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53f8157470 0x7f53f80c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:59.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.876+0000 7f53fcc9e640 1 --2- 172.21.15.131:0/1081466960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53f8070550 0x7f53f8156cc0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:59.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.876+0000 7f53fcc9e640 1 --2- 172.21.15.131:0/1081466960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53f806a250 0x7f53f81579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:55:59.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.876+0000 7f53fcc9e640 1 -- 172.21.15.131:0/1081466960 >> 172.21.15.131:0/1081466960 conn(0x7f53f80cd150 msgr2=0x7f53f8158290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:55:59.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.876+0000 7f53fcc9e640 1 -- 172.21.15.131:0/1081466960 shutdown_connections 2023-09-12T14:55:59.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:55:59.876+0000 7f53fcc9e640 1 -- 172.21.15.131:0/1081466960 wait complete. 2023-09-12T14:55:59.896 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:56:00.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.008+0000 7f6df4191640 1 -- 172.21.15.131:0/2518956201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dd8005190 msgr2=0x7f6dd800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:00.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.008+0000 7f6df4191640 1 --2- 172.21.15.131:0/2518956201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dd8005190 0x7f6dd800b990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009ee0 tx=0x7f6de401d610 comp rx=0 tx=0).stop 2023-09-12T14:56:00.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.008+0000 7f6df4191640 1 -- 172.21.15.131:0/2518956201 shutdown_connections 2023-09-12T14:56:00.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.008+0000 7f6df4191640 1 --2- 172.21.15.131:0/2518956201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dd8005190 0x7f6dd800b990 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:00.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.008+0000 7f6df4191640 1 -- 172.21.15.131:0/2518956201 >> 172.21.15.131:0/2518956201 conn(0x7f6dec069840 msgr2=0x7f6dec0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:00.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.008+0000 7f6df4191640 1 -- 172.21.15.131:0/2518956201 shutdown_connections 2023-09-12T14:56:00.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.008+0000 7f6df4191640 1 -- 172.21.15.131:0/2518956201 wait complete. 2023-09-12T14:56:00.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.012+0000 7f6df4191640 1 Processor -- start 2023-09-12T14:56:00.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.012+0000 7f6df4191640 1 -- start start 2023-09-12T14:56:00.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.012+0000 7f6df4191640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6dd8005190 0x7f6dec13b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:00.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.012+0000 7f6df4191640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dec15ac90 0x7f6dec15a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:00.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.012+0000 7f6df4191640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6dec0ae690 0x7f6dec0aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:00.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.012+0000 7f6df4191640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec0b25d0 con 0x7f6dec15ac90 2023-09-12T14:56:00.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.012+0000 7f6df4191640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec0af030 con 0x7f6dd8005190 2023-09-12T14:56:00.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.012+0000 7f6df4191640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec1f4130 con 0x7f6dec0ae690 2023-09-12T14:56:00.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.012+0000 7f6df172c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dec15ac90 0x7f6dec15a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:00.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.012+0000 7f6df172c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dec15ac90 0x7f6dec15a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37350/0 (socket says 172.21.15.131:37350) 2023-09-12T14:56:00.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.012+0000 7f6df172c640 1 -- 172.21.15.131:0/3403688385 learned_addr learned my addr 172.21.15.131:0/3403688385 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:00.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.012+0000 7f6df1f2d640 1 --2- 172.21.15.131:0/3403688385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6dd8005190 0x7f6dec13b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:00.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.012+0000 7f6df272e640 1 --2- 172.21.15.131:0/3403688385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6dec0ae690 0x7f6dec0aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:00.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.012+0000 7f6df172c640 1 -- 172.21.15.131:0/3403688385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6dec0ae690 msgr2=0x7f6dec0aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:00.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.012+0000 7f6df172c640 1 --2- 172.21.15.131:0/3403688385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6dec0ae690 0x7f6dec0aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:00.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.012+0000 7f6df172c640 1 -- 172.21.15.131:0/3403688385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6dd8005190 msgr2=0x7f6dec13b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:00.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.012+0000 7f6df172c640 1 --2- 172.21.15.131:0/3403688385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6dd8005190 0x7f6dec13b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:00.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.012+0000 7f6df172c640 1 -- 172.21.15.131:0/3403688385 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dec15b110 con 0x7f6dec15ac90 2023-09-12T14:56:00.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.012+0000 7f6df172c640 1 --2- 172.21.15.131:0/3403688385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dec15ac90 0x7f6dec15a420 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc004da0 tx=0x7f6ddc001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:00.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.012+0000 7f6de27fc640 1 -- 172.21.15.131:0/3403688385 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc011870 con 0x7f6dec15ac90 2023-09-12T14:56:00.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.012+0000 7f6de27fc640 1 -- 172.21.15.131:0/3403688385 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6ddc011ac0 con 0x7f6dec15ac90 2023-09-12T14:56:00.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.012+0000 7f6df4191640 1 -- 172.21.15.131:0/3403688385 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dec1f4380 con 0x7f6dec15ac90 2023-09-12T14:56:00.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.012+0000 7f6de27fc640 1 -- 172.21.15.131:0/3403688385 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc001d50 con 0x7f6dec15ac90 2023-09-12T14:56:00.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.012+0000 7f6df4191640 1 -- 172.21.15.131:0/3403688385 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dec1f4940 con 0x7f6dec15ac90 2023-09-12T14:56:00.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.012+0000 7f6df4191640 1 -- 172.21.15.131:0/3403688385 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dec073580 con 0x7f6dec15ac90 2023-09-12T14:56:00.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.012+0000 7f6de27fc640 1 -- 172.21.15.131:0/3403688385 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f6ddc01a400 con 0x7f6dec15ac90 2023-09-12T14:56:00.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.012+0000 7f6de27fc640 1 --2- 172.21.15.131:0/3403688385 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6dd0065b50 0x7f6dd0068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:00.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.016+0000 7f6de27fc640 1 -- 172.21.15.131:0/3403688385 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ddc083e40 con 0x7f6dec15ac90 2023-09-12T14:56:00.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.016+0000 7f6df1f2d640 1 --2- 172.21.15.131:0/3403688385 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6dd0065b50 0x7f6dd0068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:00.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.016+0000 7f6df1f2d640 1 --2- 172.21.15.131:0/3403688385 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6dd0065b50 0x7f6dd0068010 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6dec1f4090 tx=0x7f6de80113d0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:00.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.020+0000 7f6de27fc640 1 -- 172.21.15.131:0/3403688385 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f6ddc0518e0 con 0x7f6dec15ac90 2023-09-12T14:56:00.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.172+0000 7f6df4191640 1 -- 172.21.15.131:0/3403688385 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6dec06f950 con 0x7f6dec15ac90 2023-09-12T14:56:00.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.176+0000 7f6de27fc640 1 -- 172.21.15.131:0/3403688385 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f6ddc054f00 con 0x7f6dec15ac90 2023-09-12T14:56:00.178 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:56:00.178 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":33,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:55:57.898819+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:55:55.034593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-09-12T14:55:55.855927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:56:00.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.176+0000 7f6df4191640 1 -- 172.21.15.131:0/3403688385 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6dd0065b50 msgr2=0x7f6dd0068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:00.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.176+0000 7f6df4191640 1 --2- 172.21.15.131:0/3403688385 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6dd0065b50 0x7f6dd0068010 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6dec1f4090 tx=0x7f6de80113d0 comp rx=0 tx=0).stop 2023-09-12T14:56:00.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.176+0000 7f6df4191640 1 -- 172.21.15.131:0/3403688385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dec15ac90 msgr2=0x7f6dec15a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:00.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.176+0000 7f6df4191640 1 --2- 172.21.15.131:0/3403688385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dec15ac90 0x7f6dec15a420 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc004da0 tx=0x7f6ddc001640 comp rx=0 tx=0).stop 2023-09-12T14:56:00.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.176+0000 7f6df4191640 1 -- 172.21.15.131:0/3403688385 shutdown_connections 2023-09-12T14:56:00.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.176+0000 7f6df4191640 1 --2- 172.21.15.131:0/3403688385 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6dd0065b50 0x7f6dd0068010 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:00.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.176+0000 7f6df4191640 1 --2- 172.21.15.131:0/3403688385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6dec0ae690 0x7f6dec0aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:00.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.176+0000 7f6df4191640 1 --2- 172.21.15.131:0/3403688385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dec15ac90 0x7f6dec15a420 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:00.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.176+0000 7f6df4191640 1 --2- 172.21.15.131:0/3403688385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6dd8005190 0x7f6dec13b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:00.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.176+0000 7f6df4191640 1 -- 172.21.15.131:0/3403688385 >> 172.21.15.131:0/3403688385 conn(0x7f6dec069840 msgr2=0x7f6dec06fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:00.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.176+0000 7f6df4191640 1 -- 172.21.15.131:0/3403688385 shutdown_connections 2023-09-12T14:56:00.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.176+0000 7f6df4191640 1 -- 172.21.15.131:0/3403688385 wait complete. 2023-09-12T14:56:00.196 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-12T14:56:00.196 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-12T14:56:00.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.320+0000 7fb936d2e640 1 -- 172.21.15.131:0/2771005040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9200047d0 msgr2=0x7fb920004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:00.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.320+0000 7fb936d2e640 1 --2- 172.21.15.131:0/2771005040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9200047d0 0x7fb920004c50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0017c0 tx=0x7fb91c01f010 comp rx=0 tx=0).stop 2023-09-12T14:56:00.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.320+0000 7fb9352cb640 1 -- 172.21.15.131:0/2771005040 reap_dead start 2023-09-12T14:56:00.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.320+0000 7fb936d2e640 1 -- 172.21.15.131:0/2771005040 shutdown_connections 2023-09-12T14:56:00.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.320+0000 7fb936d2e640 1 -- 172.21.15.131:0/2771005040 >> 172.21.15.131:0/2771005040 conn(0x7fb93006b310 msgr2=0x7fb93006b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:00.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.320+0000 7fb936d2e640 1 -- 172.21.15.131:0/2771005040 shutdown_connections 2023-09-12T14:56:00.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.320+0000 7fb936d2e640 1 -- 172.21.15.131:0/2771005040 wait complete. 2023-09-12T14:56:00.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.320+0000 7fb936d2e640 1 Processor -- start 2023-09-12T14:56:00.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.320+0000 7fb936d2e640 1 -- start start 2023-09-12T14:56:00.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb936d2e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9200047d0 0x7fb9300c0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:00.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb936d2e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb93015aab0 0x7fb93014dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:00.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb936d2e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb93014e550 0x7fb9300ab6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:00.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb936d2e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93015af90 con 0x7fb9200047d0 2023-09-12T14:56:00.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb936d2e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9301551d0 con 0x7fb93015aab0 2023-09-12T14:56:00.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb936d2e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9301554b0 con 0x7fb93014e550 2023-09-12T14:56:00.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb934aca640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9200047d0 0x7fb9300c0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:00.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb934aca640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9200047d0 0x7fb9300c0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37368/0 (socket says 172.21.15.131:37368) 2023-09-12T14:56:00.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb934aca640 1 -- 172.21.15.131:0/3344182110 learned_addr learned my addr 172.21.15.131:0/3344182110 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:00.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb92ffff640 1 --2- 172.21.15.131:0/3344182110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb93015aab0 0x7fb93014dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:00.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb9352cb640 1 --2- 172.21.15.131:0/3344182110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb93014e550 0x7fb9300ab6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:00.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb934aca640 1 -- 172.21.15.131:0/3344182110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb93014e550 msgr2=0x7fb9300ab6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:00.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb934aca640 1 --2- 172.21.15.131:0/3344182110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb93014e550 0x7fb9300ab6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:00.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb934aca640 1 -- 172.21.15.131:0/3344182110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb93015aab0 msgr2=0x7fb93014dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:00.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb934aca640 1 --2- 172.21.15.131:0/3344182110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb93015aab0 0x7fb93014dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:00.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb934aca640 1 -- 172.21.15.131:0/3344182110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9300c1010 con 0x7fb9200047d0 2023-09-12T14:56:00.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb92ffff640 1 --2- 172.21.15.131:0/3344182110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb93015aab0 0x7fb93014dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:56:00.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb934aca640 1 --2- 172.21.15.131:0/3344182110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9200047d0 0x7fb9300c0ad0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb924011a50 tx=0x7fb924011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:00.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb92d7fa640 1 -- 172.21.15.131:0/3344182110 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92400fd90 con 0x7fb9200047d0 2023-09-12T14:56:00.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb92d7fa640 1 -- 172.21.15.131:0/3344182110 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb924012450 con 0x7fb9200047d0 2023-09-12T14:56:00.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb92d7fa640 1 -- 172.21.15.131:0/3344182110 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb924012800 con 0x7fb9200047d0 2023-09-12T14:56:00.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb936d2e640 1 -- 172.21.15.131:0/3344182110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb93014e1b0 con 0x7fb9200047d0 2023-09-12T14:56:00.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb936d2e640 1 -- 172.21.15.131:0/3344182110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9300abdf0 con 0x7fb9200047d0 2023-09-12T14:56:00.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb936d2e640 1 -- 172.21.15.131:0/3344182110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb930155030 con 0x7fb9200047d0 2023-09-12T14:56:00.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb92d7fa640 1 -- 172.21.15.131:0/3344182110 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fb924012960 con 0x7fb9200047d0 2023-09-12T14:56:00.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb92d7fa640 1 --2- 172.21.15.131:0/3344182110 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb914065b20 0x7fb914067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:00.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb92d7fa640 1 -- 172.21.15.131:0/3344182110 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb924084270 con 0x7fb9200047d0 2023-09-12T14:56:00.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb92ffff640 1 --2- 172.21.15.131:0/3344182110 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb914065b20 0x7fb914067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:00.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.324+0000 7fb92ffff640 1 --2- 172.21.15.131:0/3344182110 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb914065b20 0x7fb914067fe0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb91c001790 tx=0x7fb91c0aa040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:00.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.328+0000 7fb92d7fa640 1 -- 172.21.15.131:0/3344182110 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fb930155030 con 0x7fb9200047d0 2023-09-12T14:56:00.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.472+0000 7fb936d2e640 1 -- 172.21.15.131:0/3344182110 --> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb930155340 con 0x7fb914065b20 2023-09-12T14:56:00.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.484+0000 7fb92d7fa640 1 -- 172.21.15.131:0/3344182110 <== mgr.4200 v2:172.21.15.131:6832/2298103651 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+275756 (secure 0 0 0) 0x7fb930155340 con 0x7fb914065b20 2023-09-12T14:56:00.488 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:56:00.489 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-09-12T14:56:00.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.484+0000 7fb936d2e640 1 -- 172.21.15.131:0/3344182110 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb914065b20 msgr2=0x7fb914067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:00.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.484+0000 7fb936d2e640 1 --2- 172.21.15.131:0/3344182110 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb914065b20 0x7fb914067fe0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb91c001790 tx=0x7fb91c0aa040 comp rx=0 tx=0).stop 2023-09-12T14:56:00.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.484+0000 7fb936d2e640 1 -- 172.21.15.131:0/3344182110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9200047d0 msgr2=0x7fb9300c0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:00.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.484+0000 7fb936d2e640 1 --2- 172.21.15.131:0/3344182110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9200047d0 0x7fb9300c0ad0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb924011a50 tx=0x7fb924011f10 comp rx=0 tx=0).stop 2023-09-12T14:56:00.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.484+0000 7fb936d2e640 1 -- 172.21.15.131:0/3344182110 shutdown_connections 2023-09-12T14:56:00.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.484+0000 7fb936d2e640 1 --2- 172.21.15.131:0/3344182110 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb914065b20 0x7fb914067fe0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:00.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.484+0000 7fb936d2e640 1 --2- 172.21.15.131:0/3344182110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb93014e550 0x7fb9300ab6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:00.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.484+0000 7fb936d2e640 1 --2- 172.21.15.131:0/3344182110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb93015aab0 0x7fb93014dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:00.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.484+0000 7fb936d2e640 1 --2- 172.21.15.131:0/3344182110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9200047d0 0x7fb9300c0ad0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:00.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.484+0000 7fb936d2e640 1 -- 172.21.15.131:0/3344182110 >> 172.21.15.131:0/3344182110 conn(0x7fb93006b310 msgr2=0x7fb9300ad550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:00.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.484+0000 7fb936d2e640 1 -- 172.21.15.131:0/3344182110 shutdown_connections 2023-09-12T14:56:00.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.484+0000 7fb936d2e640 1 -- 172.21.15.131:0/3344182110 wait complete. 2023-09-12T14:56:00.504 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-09-12T14:55:59.595951+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219924,"kb_used_data":3696,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754796,"statfs":{"total":773094113280,"available":772868911104,"internally_reserved":0,"allocated":3784704,"data_stored":2159686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.101560"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.506038+0000","last_change":"2023-09-12T14:55:55.506038+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.506038+0000","last_undegraded":"2023-09-12T14:55:55.506038+0000","last_fullsized":"2023-09-12T14:55:55.506038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.501947+0000","last_change":"2023-09-12T14:55:55.501947+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.501947+0000","last_undegraded":"2023-09-12T14:55:55.501947+0000","last_fullsized":"2023-09-12T14:55:55.501947+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.503426+0000","last_change":"2023-09-12T14:55:55.503426+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.503426+0000","last_undegraded":"2023-09-12T14:55:55.503426+0000","last_fullsized":"2023-09-12T14:55:55.503426+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.506221+0000","last_change":"2023-09-12T14:55:55.506221+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.506221+0000","last_undegraded":"2023-09-12T14:55:55.506221+0000","last_fullsized":"2023-09-12T14:55:55.506221+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.505165+0000","last_change":"2023-09-12T14:55:55.505165+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.505165+0000","last_undegraded":"2023-09-12T14:55:55.505165+0000","last_fullsized":"2023-09-12T14:55:55.505165+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.499803+0000","last_change":"2023-09-12T14:55:55.499803+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.499803+0000","last_undegraded":"2023-09-12T14:55:55.499803+0000","last_fullsized":"2023-09-12T14:55:55.499803+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.504192+0000","last_change":"2023-09-12T14:55:55.504192+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.504192+0000","last_undegraded":"2023-09-12T14:55:55.504192+0000","last_fullsized":"2023-09-12T14:55:55.504192+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.499816+0000","last_change":"2023-09-12T14:55:55.499816+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.499816+0000","last_undegraded":"2023-09-12T14:55:55.499816+0000","last_fullsized":"2023-09-12T14:55:55.499816+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.504086+0000","last_change":"2023-09-12T14:55:55.504086+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.504086+0000","last_undegraded":"2023-09-12T14:55:55.504086+0000","last_fullsized":"2023-09-12T14:55:55.504086+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.501264+0000","last_change":"2023-09-12T14:55:55.501264+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.501264+0000","last_undegraded":"2023-09-12T14:55:55.501264+0000","last_fullsized":"2023-09-12T14:55:55.501264+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.500589+0000","last_change":"2023-09-12T14:55:55.500589+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.500589+0000","last_undegraded":"2023-09-12T14:55:55.500589+0000","last_fullsized":"2023-09-12T14:55:55.500589+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.503522+0000","last_change":"2023-09-12T14:55:55.503522+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.503522+0000","last_undegraded":"2023-09-12T14:55:55.503522+0000","last_fullsized":"2023-09-12T14:55:55.503522+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.502591+0000","last_change":"2023-09-12T14:55:55.502591+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.502591+0000","last_undegraded":"2023-09-12T14:55:55.502591+0000","last_fullsized":"2023-09-12T14:55:55.502591+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.503708+0000","last_change":"2023-09-12T14:55:55.503708+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.503708+0000","last_undegraded":"2023-09-12T14:55:55.503708+0000","last_fullsized":"2023-09-12T14:55:55.503708+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.506010+0000","last_change":"2023-09-12T14:55:55.506010+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.506010+0000","last_undegraded":"2023-09-12T14:55:55.506010+0000","last_fullsized":"2023-09-12T14:55:55.506010+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.503103+0000","last_change":"2023-09-12T14:55:55.503103+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.503103+0000","last_undegraded":"2023-09-12T14:55:55.503103+0000","last_fullsized":"2023-09-12T14:55:55.503103+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.503457+0000","last_change":"2023-09-12T14:55:55.503457+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.503457+0000","last_undegraded":"2023-09-12T14:55:55.503457+0000","last_fullsized":"2023-09-12T14:55:55.503457+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.500630+0000","last_change":"2023-09-12T14:55:55.500630+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.500630+0000","last_undegraded":"2023-09-12T14:55:55.500630+0000","last_fullsized":"2023-09-12T14:55:55.500630+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.499990+0000","last_change":"2023-09-12T14:55:55.499990+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.499990+0000","last_undegraded":"2023-09-12T14:55:55.499990+0000","last_fullsized":"2023-09-12T14:55:55.499990+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.504226+0000","last_change":"2023-09-12T14:55:55.504226+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.504226+0000","last_undegraded":"2023-09-12T14:55:55.504226+0000","last_fullsized":"2023-09-12T14:55:55.504226+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.500759+0000","last_change":"2023-09-12T14:55:55.500759+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.500759+0000","last_undegraded":"2023-09-12T14:55:55.500759+0000","last_fullsized":"2023-09-12T14:55:55.500759+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.506589+0000","last_change":"2023-09-12T14:55:55.506589+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.506589+0000","last_undegraded":"2023-09-12T14:55:55.506589+0000","last_fullsized":"2023-09-12T14:55:55.506589+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.504932+0000","last_change":"2023-09-12T14:55:55.504932+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.504932+0000","last_undegraded":"2023-09-12T14:55:55.504932+0000","last_fullsized":"2023-09-12T14:55:55.504932+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.499587+0000","last_change":"2023-09-12T14:55:55.499587+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.499587+0000","last_undegraded":"2023-09-12T14:55:55.499587+0000","last_fullsized":"2023-09-12T14:55:55.499587+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.502791+0000","last_change":"2023-09-12T14:55:55.502791+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.502791+0000","last_undegraded":"2023-09-12T14:55:55.502791+0000","last_fullsized":"2023-09-12T14:55:55.502791+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.499996+0000","last_change":"2023-09-12T14:55:55.499996+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.499996+0000","last_undegraded":"2023-09-12T14:55:55.499996+0000","last_fullsized":"2023-09-12T14:55:55.499996+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.503433+0000","last_change":"2023-09-12T14:55:55.503433+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.503433+0000","last_undegraded":"2023-09-12T14:55:55.503433+0000","last_fullsized":"2023-09-12T14:55:55.503433+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.499833+0000","last_change":"2023-09-12T14:55:55.499833+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.499833+0000","last_undegraded":"2023-09-12T14:55:55.499833+0000","last_fullsized":"2023-09-12T14:55:55.499833+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.503026+0000","last_change":"2023-09-12T14:55:55.503026+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.503026+0000","last_undegraded":"2023-09-12T14:55:55.503026+0000","last_fullsized":"2023-09-12T14:55:55.503026+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.503582+0000","last_change":"2023-09-12T14:55:55.503582+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.503582+0000","last_undegraded":"2023-09-12T14:55:55.503582+0000","last_fullsized":"2023-09-12T14:55:55.503582+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.504113+0000","last_change":"2023-09-12T14:55:55.504113+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.504113+0000","last_undegraded":"2023-09-12T14:55:55.504113+0000","last_fullsized":"2023-09-12T14:55:55.504113+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.506309+0000","last_change":"2023-09-12T14:55:55.506309+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.506309+0000","last_undegraded":"2023-09-12T14:55:55.506309+0000","last_fullsized":"2023-09-12T14:55:55.506309+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":173,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-09-12T14:55:51.718431+0000","last_change":"2023-09-12T14:54:46.965446+0000","last_active":"2023-09-12T14:55:51.718431+0000","last_peered":"2023-09-12T14:55:51.718431+0000","last_clean":"2023-09-12T14:55:51.718431+0000","last_became_active":"2023-09-12T14:54:46.963236+0000","last_became_peered":"2023-09-12T14:54:46.963236+0000","last_unstale":"2023-09-12T14:55:51.718431+0000","last_undegraded":"2023-09-12T14:55:51.718431+0000","last_fullsized":"2023-09-12T14:55:51.718431+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:45.804232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:45.804232+0000","last_clean_scrub_stamp":"2023-09-12T14:54:45.804232+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:24:45.670513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":917504,"data_stored":712137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":1.244},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":1.351}]},{"osd":1,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":1.2949999999999999}]},{"osd":2,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.073}]},{"osd":3,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.115},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.94399999999999995}]},{"osd":4,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.99299999999999999}]},{"osd":5,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.159}]},{"osd":6,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.04},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.2050000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":323584,"data_stored":121769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.65300000000000002}]},{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.84399999999999997}]},{"osd":2,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":1.0800000000000001}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.91300000000000003}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":1.133}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":1.113},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":1.1910000000000001}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":1.256}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":323584,"data_stored":121769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.94199999999999995}]},{"osd":1,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.64400000000000002}]},{"osd":2,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.56999999999999995}]},{"osd":3,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.753}]},{"osd":4,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.0900000000000001}]},{"osd":6,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.516,"5min":2.516,"15min":2.516},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.85199999999999998}]},{"osd":7,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.135},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":1}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":323584,"data_stored":121769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.81299999999999994}]},{"osd":1,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.86799999999999999}]},{"osd":2,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":0.94199999999999995}]},{"osd":3,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":1.1659999999999999}]},{"osd":5,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":1.044}]},{"osd":6,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":1.069},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":1.093}]},{"osd":7,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":1.222}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.31900000000000001}]},{"osd":1,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.35999999999999999}]},{"osd":2,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.30199999999999999}]},{"osd":4,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.76400000000000001}]},{"osd":5,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.64600000000000002}]},{"osd":6,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.61599999999999999}]},{"osd":7,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":0.626},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.81100000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":323584,"data_stored":121769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.47899999999999998}]},{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.441}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.41899999999999998}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.56399999999999995}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.60899999999999999}]},{"osd":6,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.1989999999999998,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":0.69099999999999995}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.63500000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":323584,"data_stored":121769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.627},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.78800000000000003}]},{"osd":2,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.499},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.85499999999999998}]},{"osd":3,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.755},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.54400000000000004}]},{"osd":4,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.94099999999999995}]},{"osd":5,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.879},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.95799999999999996}]},{"osd":6,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":1.105},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.91800000000000004}]},{"osd":7,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.0589999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":913408,"data_stored":711118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.66200000000000003}]},{"osd":2,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.76800000000000002}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":1.1519999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":1.127}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.054}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":1.091},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":0.94499999999999995}]},{"osd":6,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.1719999999999999}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.196}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-12T14:56:00.510 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-12T14:56:00.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.668+0000 7f1913bb3640 1 -- 172.21.15.131:0/4214264530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f40047d0 msgr2=0x7f18f4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:00.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.668+0000 7f1913bb3640 1 --2- 172.21.15.131:0/4214264530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f40047d0 0x7f18f4004c50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1900001680 tx=0x7f190001c780 comp rx=0 tx=0).stop 2023-09-12T14:56:00.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.668+0000 7f1913bb3640 1 -- 172.21.15.131:0/4214264530 shutdown_connections 2023-09-12T14:56:00.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.668+0000 7f1913bb3640 1 --2- 172.21.15.131:0/4214264530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f40047d0 0x7f18f4004c50 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:00.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.668+0000 7f1913bb3640 1 -- 172.21.15.131:0/4214264530 >> 172.21.15.131:0/4214264530 conn(0x7f190c069840 msgr2=0x7f190c0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:00.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.668+0000 7f1913bb3640 1 -- 172.21.15.131:0/4214264530 shutdown_connections 2023-09-12T14:56:00.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.668+0000 7f1913bb3640 1 -- 172.21.15.131:0/4214264530 wait complete. 2023-09-12T14:56:00.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.668+0000 7f1913bb3640 1 Processor -- start 2023-09-12T14:56:00.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.668+0000 7f1913bb3640 1 -- start start 2023-09-12T14:56:00.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.668+0000 7f1913bb3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18f40047d0 0x7f190c14dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:00.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.668+0000 7f1913bb3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f190c0a64d0 0x7f190c1e1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:00.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.668+0000 7f1913bb3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f190c1e22d0 0x7f190c0a9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:00.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.668+0000 7f1913bb3640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190c0a7d00 con 0x7f190c1e22d0 2023-09-12T14:56:00.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.668+0000 7f1913bb3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190c0a9170 con 0x7f18f40047d0 2023-09-12T14:56:00.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.668+0000 7f1913bb3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190c0a7740 con 0x7f190c0a64d0 2023-09-12T14:56:00.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.668+0000 7f1912150640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f190c1e22d0 0x7f190c0a9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:00.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.668+0000 7f1912150640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f190c1e22d0 0x7f190c0a9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37382/0 (socket says 172.21.15.131:37382) 2023-09-12T14:56:00.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.668+0000 7f1912150640 1 -- 172.21.15.131:0/4259166568 learned_addr learned my addr 172.21.15.131:0/4259166568 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:00.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.668+0000 7f191114e640 1 --2- 172.21.15.131:0/4259166568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f190c0a64d0 0x7f190c1e1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:00.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.668+0000 7f191194f640 1 --2- 172.21.15.131:0/4259166568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18f40047d0 0x7f190c14dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:00.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.672+0000 7f1912150640 1 -- 172.21.15.131:0/4259166568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f190c0a64d0 msgr2=0x7f190c1e1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:00.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.672+0000 7f1912150640 1 --2- 172.21.15.131:0/4259166568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f190c0a64d0 0x7f190c1e1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:00.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.672+0000 7f1912150640 1 -- 172.21.15.131:0/4259166568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18f40047d0 msgr2=0x7f190c14dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:00.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.672+0000 7f1912150640 1 --2- 172.21.15.131:0/4259166568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18f40047d0 0x7f190c14dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:00.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.672+0000 7f1912150640 1 -- 172.21.15.131:0/4259166568 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190c0a7a20 con 0x7f190c1e22d0 2023-09-12T14:56:00.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.672+0000 7f191194f640 1 --2- 172.21.15.131:0/4259166568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18f40047d0 0x7f190c14dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:00.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.672+0000 7f1912150640 1 --2- 172.21.15.131:0/4259166568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f190c1e22d0 0x7f190c0a9c10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f190c0cc890 tx=0x7f18fc00f8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:00.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.672+0000 7f190a7fc640 1 -- 172.21.15.131:0/4259166568 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc0080d0 con 0x7f190c1e22d0 2023-09-12T14:56:00.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.672+0000 7f190a7fc640 1 -- 172.21.15.131:0/4259166568 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f18fc008250 con 0x7f190c1e22d0 2023-09-12T14:56:00.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.672+0000 7f1913bb3640 1 -- 172.21.15.131:0/4259166568 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190c0a9410 con 0x7f190c1e22d0 2023-09-12T14:56:00.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.672+0000 7f190a7fc640 1 -- 172.21.15.131:0/4259166568 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc008660 con 0x7f190c1e22d0 2023-09-12T14:56:00.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.672+0000 7f1913bb3640 1 -- 172.21.15.131:0/4259166568 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190c14e4e0 con 0x7f190c1e22d0 2023-09-12T14:56:00.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.672+0000 7f1913bb3640 1 -- 172.21.15.131:0/4259166568 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f190c073580 con 0x7f190c1e22d0 2023-09-12T14:56:00.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.672+0000 7f190a7fc640 1 -- 172.21.15.131:0/4259166568 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f18fc0087c0 con 0x7f190c1e22d0 2023-09-12T14:56:00.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.672+0000 7f190a7fc640 1 --2- 172.21.15.131:0/4259166568 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f18f8065b20 0x7f18f8067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:00.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.672+0000 7f191194f640 1 --2- 172.21.15.131:0/4259166568 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f18f8065b20 0x7f18f8067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:00.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.672+0000 7f190a7fc640 1 -- 172.21.15.131:0/4259166568 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18fc085840 con 0x7f190c1e22d0 2023-09-12T14:56:00.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.672+0000 7f191194f640 1 --2- 172.21.15.131:0/4259166568 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f18f8065b20 0x7f18f8067fe0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1904009b20 tx=0x7f19040091e0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:00.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.676+0000 7f190a7fc640 1 -- 172.21.15.131:0/4259166568 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f18fc053220 con 0x7f190c1e22d0 2023-09-12T14:56:00.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.828+0000 7f1913bb3640 1 -- 172.21.15.131:0/4259166568 --> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f190c1524a0 con 0x7f18f8065b20 2023-09-12T14:56:00.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.836+0000 7f190a7fc640 1 -- 172.21.15.131:0/4259166568 <== mgr.4200 v2:172.21.15.131:6832/2298103651 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+275756 (secure 0 0 0) 0x7f190401b440 con 0x7f18f8065b20 2023-09-12T14:56:00.841 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:56:00.845 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-09-12T14:56:00.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.840+0000 7f1913bb3640 1 -- 172.21.15.131:0/4259166568 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f18f8065b20 msgr2=0x7f18f8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:00.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.840+0000 7f1913bb3640 1 --2- 172.21.15.131:0/4259166568 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f18f8065b20 0x7f18f8067fe0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1904009b20 tx=0x7f19040091e0 comp rx=0 tx=0).stop 2023-09-12T14:56:00.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.840+0000 7f1913bb3640 1 -- 172.21.15.131:0/4259166568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f190c1e22d0 msgr2=0x7f190c0a9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:00.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.840+0000 7f1913bb3640 1 --2- 172.21.15.131:0/4259166568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f190c1e22d0 0x7f190c0a9c10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f190c0cc890 tx=0x7f18fc00f8e0 comp rx=0 tx=0).stop 2023-09-12T14:56:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.844+0000 7f1913bb3640 1 -- 172.21.15.131:0/4259166568 shutdown_connections 2023-09-12T14:56:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.844+0000 7f1913bb3640 1 --2- 172.21.15.131:0/4259166568 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f18f8065b20 0x7f18f8067fe0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.844+0000 7f1913bb3640 1 --2- 172.21.15.131:0/4259166568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f190c0a64d0 0x7f190c1e1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.844+0000 7f1913bb3640 1 --2- 172.21.15.131:0/4259166568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f190c1e22d0 0x7f190c0a9c10 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.844+0000 7f1913bb3640 1 --2- 172.21.15.131:0/4259166568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18f40047d0 0x7f190c14dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:00.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.844+0000 7f1913bb3640 1 -- 172.21.15.131:0/4259166568 >> 172.21.15.131:0/4259166568 conn(0x7f190c069840 msgr2=0x7f190c1502a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:00.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.844+0000 7f1913bb3640 1 -- 172.21.15.131:0/4259166568 shutdown_connections 2023-09-12T14:56:00.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:00.844+0000 7f1913bb3640 1 -- 172.21.15.131:0/4259166568 wait complete. 2023-09-12T14:56:00.862 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-09-12T14:55:59.595951+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219924,"kb_used_data":3696,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754796,"statfs":{"total":773094113280,"available":772868911104,"internally_reserved":0,"allocated":3784704,"data_stored":2159686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.101560"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.506038+0000","last_change":"2023-09-12T14:55:55.506038+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.506038+0000","last_undegraded":"2023-09-12T14:55:55.506038+0000","last_fullsized":"2023-09-12T14:55:55.506038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.501947+0000","last_change":"2023-09-12T14:55:55.501947+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.501947+0000","last_undegraded":"2023-09-12T14:55:55.501947+0000","last_fullsized":"2023-09-12T14:55:55.501947+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.503426+0000","last_change":"2023-09-12T14:55:55.503426+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.503426+0000","last_undegraded":"2023-09-12T14:55:55.503426+0000","last_fullsized":"2023-09-12T14:55:55.503426+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.506221+0000","last_change":"2023-09-12T14:55:55.506221+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.506221+0000","last_undegraded":"2023-09-12T14:55:55.506221+0000","last_fullsized":"2023-09-12T14:55:55.506221+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.505165+0000","last_change":"2023-09-12T14:55:55.505165+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.505165+0000","last_undegraded":"2023-09-12T14:55:55.505165+0000","last_fullsized":"2023-09-12T14:55:55.505165+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.499803+0000","last_change":"2023-09-12T14:55:55.499803+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.499803+0000","last_undegraded":"2023-09-12T14:55:55.499803+0000","last_fullsized":"2023-09-12T14:55:55.499803+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.504192+0000","last_change":"2023-09-12T14:55:55.504192+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.504192+0000","last_undegraded":"2023-09-12T14:55:55.504192+0000","last_fullsized":"2023-09-12T14:55:55.504192+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.499816+0000","last_change":"2023-09-12T14:55:55.499816+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.499816+0000","last_undegraded":"2023-09-12T14:55:55.499816+0000","last_fullsized":"2023-09-12T14:55:55.499816+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.504086+0000","last_change":"2023-09-12T14:55:55.504086+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.504086+0000","last_undegraded":"2023-09-12T14:55:55.504086+0000","last_fullsized":"2023-09-12T14:55:55.504086+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.501264+0000","last_change":"2023-09-12T14:55:55.501264+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.501264+0000","last_undegraded":"2023-09-12T14:55:55.501264+0000","last_fullsized":"2023-09-12T14:55:55.501264+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.500589+0000","last_change":"2023-09-12T14:55:55.500589+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.500589+0000","last_undegraded":"2023-09-12T14:55:55.500589+0000","last_fullsized":"2023-09-12T14:55:55.500589+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.503522+0000","last_change":"2023-09-12T14:55:55.503522+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.503522+0000","last_undegraded":"2023-09-12T14:55:55.503522+0000","last_fullsized":"2023-09-12T14:55:55.503522+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.502591+0000","last_change":"2023-09-12T14:55:55.502591+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.502591+0000","last_undegraded":"2023-09-12T14:55:55.502591+0000","last_fullsized":"2023-09-12T14:55:55.502591+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.503708+0000","last_change":"2023-09-12T14:55:55.503708+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.503708+0000","last_undegraded":"2023-09-12T14:55:55.503708+0000","last_fullsized":"2023-09-12T14:55:55.503708+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.506010+0000","last_change":"2023-09-12T14:55:55.506010+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.506010+0000","last_undegraded":"2023-09-12T14:55:55.506010+0000","last_fullsized":"2023-09-12T14:55:55.506010+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.503103+0000","last_change":"2023-09-12T14:55:55.503103+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.503103+0000","last_undegraded":"2023-09-12T14:55:55.503103+0000","last_fullsized":"2023-09-12T14:55:55.503103+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.503457+0000","last_change":"2023-09-12T14:55:55.503457+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.503457+0000","last_undegraded":"2023-09-12T14:55:55.503457+0000","last_fullsized":"2023-09-12T14:55:55.503457+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.500630+0000","last_change":"2023-09-12T14:55:55.500630+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.500630+0000","last_undegraded":"2023-09-12T14:55:55.500630+0000","last_fullsized":"2023-09-12T14:55:55.500630+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.499990+0000","last_change":"2023-09-12T14:55:55.499990+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.499990+0000","last_undegraded":"2023-09-12T14:55:55.499990+0000","last_fullsized":"2023-09-12T14:55:55.499990+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.504226+0000","last_change":"2023-09-12T14:55:55.504226+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.504226+0000","last_undegraded":"2023-09-12T14:55:55.504226+0000","last_fullsized":"2023-09-12T14:55:55.504226+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.500759+0000","last_change":"2023-09-12T14:55:55.500759+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.500759+0000","last_undegraded":"2023-09-12T14:55:55.500759+0000","last_fullsized":"2023-09-12T14:55:55.500759+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.506589+0000","last_change":"2023-09-12T14:55:55.506589+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.506589+0000","last_undegraded":"2023-09-12T14:55:55.506589+0000","last_fullsized":"2023-09-12T14:55:55.506589+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.504932+0000","last_change":"2023-09-12T14:55:55.504932+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.504932+0000","last_undegraded":"2023-09-12T14:55:55.504932+0000","last_fullsized":"2023-09-12T14:55:55.504932+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.499587+0000","last_change":"2023-09-12T14:55:55.499587+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.499587+0000","last_undegraded":"2023-09-12T14:55:55.499587+0000","last_fullsized":"2023-09-12T14:55:55.499587+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.502791+0000","last_change":"2023-09-12T14:55:55.502791+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.502791+0000","last_undegraded":"2023-09-12T14:55:55.502791+0000","last_fullsized":"2023-09-12T14:55:55.502791+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.499996+0000","last_change":"2023-09-12T14:55:55.499996+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.499996+0000","last_undegraded":"2023-09-12T14:55:55.499996+0000","last_fullsized":"2023-09-12T14:55:55.499996+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.503433+0000","last_change":"2023-09-12T14:55:55.503433+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.503433+0000","last_undegraded":"2023-09-12T14:55:55.503433+0000","last_fullsized":"2023-09-12T14:55:55.503433+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.499833+0000","last_change":"2023-09-12T14:55:55.499833+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.499833+0000","last_undegraded":"2023-09-12T14:55:55.499833+0000","last_fullsized":"2023-09-12T14:55:55.499833+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.503026+0000","last_change":"2023-09-12T14:55:55.503026+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.503026+0000","last_undegraded":"2023-09-12T14:55:55.503026+0000","last_fullsized":"2023-09-12T14:55:55.503026+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.503582+0000","last_change":"2023-09-12T14:55:55.503582+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.503582+0000","last_undegraded":"2023-09-12T14:55:55.503582+0000","last_fullsized":"2023-09-12T14:55:55.503582+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.504113+0000","last_change":"2023-09-12T14:55:55.504113+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.504113+0000","last_undegraded":"2023-09-12T14:55:55.504113+0000","last_fullsized":"2023-09-12T14:55:55.504113+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.506309+0000","last_change":"2023-09-12T14:55:55.506309+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.506309+0000","last_undegraded":"2023-09-12T14:55:55.506309+0000","last_fullsized":"2023-09-12T14:55:55.506309+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":173,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-09-12T14:55:51.718431+0000","last_change":"2023-09-12T14:54:46.965446+0000","last_active":"2023-09-12T14:55:51.718431+0000","last_peered":"2023-09-12T14:55:51.718431+0000","last_clean":"2023-09-12T14:55:51.718431+0000","last_became_active":"2023-09-12T14:54:46.963236+0000","last_became_peered":"2023-09-12T14:54:46.963236+0000","last_unstale":"2023-09-12T14:55:51.718431+0000","last_undegraded":"2023-09-12T14:55:51.718431+0000","last_fullsized":"2023-09-12T14:55:51.718431+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:45.804232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:45.804232+0000","last_clean_scrub_stamp":"2023-09-12T14:54:45.804232+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:24:45.670513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":917504,"data_stored":712137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":1.244},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":1.351}]},{"osd":1,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":1.2949999999999999}]},{"osd":2,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.073}]},{"osd":3,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.115},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.94399999999999995}]},{"osd":4,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.99299999999999999}]},{"osd":5,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.159}]},{"osd":6,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.04},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.2050000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":323584,"data_stored":121769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.65300000000000002}]},{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.84399999999999997}]},{"osd":2,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":1.0800000000000001}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.91300000000000003}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":1.133}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":1.113},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":1.1910000000000001}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":1.256}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":323584,"data_stored":121769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.94199999999999995}]},{"osd":1,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.64400000000000002}]},{"osd":2,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.56999999999999995}]},{"osd":3,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.753}]},{"osd":4,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.0900000000000001}]},{"osd":6,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.516,"5min":2.516,"15min":2.516},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.85199999999999998}]},{"osd":7,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.135},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":1}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":323584,"data_stored":121769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.81299999999999994}]},{"osd":1,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.86799999999999999}]},{"osd":2,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":0.94199999999999995}]},{"osd":3,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":1.1659999999999999}]},{"osd":5,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":1.044}]},{"osd":6,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":1.069},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":1.093}]},{"osd":7,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":1.222}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.31900000000000001}]},{"osd":1,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.35999999999999999}]},{"osd":2,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.30199999999999999}]},{"osd":4,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.76400000000000001}]},{"osd":5,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.64600000000000002}]},{"osd":6,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.61599999999999999}]},{"osd":7,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":0.626},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.81100000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":323584,"data_stored":121769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.47899999999999998}]},{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.441}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.41899999999999998}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.56399999999999995}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.60899999999999999}]},{"osd":6,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.1989999999999998,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":0.69099999999999995}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.63500000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":323584,"data_stored":121769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.627},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.78800000000000003}]},{"osd":2,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.499},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.85499999999999998}]},{"osd":3,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.755},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.54400000000000004}]},{"osd":4,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.94099999999999995}]},{"osd":5,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.879},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.95799999999999996}]},{"osd":6,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":1.105},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.91800000000000004}]},{"osd":7,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.0589999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":913408,"data_stored":711118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.66200000000000003}]},{"osd":2,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.76800000000000002}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":1.1519999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":1.127}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.054}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":1.091},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":0.94499999999999995}]},{"osd":6,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.1719999999999999}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.196}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-12T14:56:00.869 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-12T14:56:01.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.032+0000 7f74cde3c640 1 -- 172.21.15.131:0/1899036034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74c806b050 msgr2=0x7f74c806b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:01.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.032+0000 7f74cde3c640 1 --2- 172.21.15.131:0/1899036034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74c806b050 0x7f74c806b4e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009bc0 tx=0x7f74c001cb50 comp rx=0 tx=0).stop 2023-09-12T14:56:01.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.032+0000 7f74cde3c640 1 -- 172.21.15.131:0/1899036034 shutdown_connections 2023-09-12T14:56:01.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.032+0000 7f74cde3c640 1 --2- 172.21.15.131:0/1899036034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74c806b050 0x7f74c806b4e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:01.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.032+0000 7f74cde3c640 1 --2- 172.21.15.131:0/1899036034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74c80ab730 0x7f74c80abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:01.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.032+0000 7f74cde3c640 1 -- 172.21.15.131:0/1899036034 >> 172.21.15.131:0/1899036034 conn(0x7f74c8069840 msgr2=0x7f74c80cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:01.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.032+0000 7f74cde3c640 1 -- 172.21.15.131:0/1899036034 shutdown_connections 2023-09-12T14:56:01.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.032+0000 7f74cde3c640 1 -- 172.21.15.131:0/1899036034 wait complete. 2023-09-12T14:56:01.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.032+0000 7f74cde3c640 1 Processor -- start 2023-09-12T14:56:01.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.032+0000 7f74cde3c640 1 -- start start 2023-09-12T14:56:01.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.032+0000 7f74cde3c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74c806b050 0x7f74c814e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:01.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.032+0000 7f74cde3c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74c80ab730 0x7f74c8153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:01.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.032+0000 7f74cde3c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74c8158180 0x7f74c8148420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:01.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.032+0000 7f74cde3c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c814d6e0 con 0x7f74c80ab730 2023-09-12T14:56:01.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.032+0000 7f74cde3c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c814d9c0 con 0x7f74c806b050 2023-09-12T14:56:01.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.032+0000 7f74cde3c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c814dca0 con 0x7f74c8158180 2023-09-12T14:56:01.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.032+0000 7f74bffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74c80ab730 0x7f74c8153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:01.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.032+0000 7f74bffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74c80ab730 0x7f74c8153120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37388/0 (socket says 172.21.15.131:37388) 2023-09-12T14:56:01.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.032+0000 7f74bffff640 1 -- 172.21.15.131:0/136863672 learned_addr learned my addr 172.21.15.131:0/136863672 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:01.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.036+0000 7f74c77fe640 1 --2- 172.21.15.131:0/136863672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74c806b050 0x7f74c814e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:01.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.036+0000 7f74c7fff640 1 --2- 172.21.15.131:0/136863672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74c8158180 0x7f74c8148420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:01.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.036+0000 7f74bffff640 1 -- 172.21.15.131:0/136863672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74c8158180 msgr2=0x7f74c8148420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:01.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.036+0000 7f74bffff640 1 --2- 172.21.15.131:0/136863672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74c8158180 0x7f74c8148420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:01.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.036+0000 7f74bffff640 1 -- 172.21.15.131:0/136863672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74c806b050 msgr2=0x7f74c814e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:01.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.036+0000 7f74bffff640 1 --2- 172.21.15.131:0/136863672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74c806b050 0x7f74c814e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:01.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.036+0000 7f74bffff640 1 -- 172.21.15.131:0/136863672 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c0009820 con 0x7f74c80ab730 2023-09-12T14:56:01.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.036+0000 7f74c77fe640 1 --2- 172.21.15.131:0/136863672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74c806b050 0x7f74c814e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:56:01.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.036+0000 7f74bffff640 1 --2- 172.21.15.131:0/136863672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74c80ab730 0x7f74c8153120 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f74b000c3c0 tx=0x7f74b000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:01.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.036+0000 7f74c4ff9640 1 -- 172.21.15.131:0/136863672 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b00052d0 con 0x7f74c80ab730 2023-09-12T14:56:01.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.036+0000 7f74c4ff9640 1 -- 172.21.15.131:0/136863672 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f74b0003600 con 0x7f74c80ab730 2023-09-12T14:56:01.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.036+0000 7f74cde3c640 1 -- 172.21.15.131:0/136863672 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c814df80 con 0x7f74c80ab730 2023-09-12T14:56:01.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.036+0000 7f74c4ff9640 1 -- 172.21.15.131:0/136863672 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b0003a10 con 0x7f74c80ab730 2023-09-12T14:56:01.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.036+0000 7f74cde3c640 1 -- 172.21.15.131:0/136863672 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c8148960 con 0x7f74c80ab730 2023-09-12T14:56:01.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.036+0000 7f74cde3c640 1 -- 172.21.15.131:0/136863672 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c8073580 con 0x7f74c80ab730 2023-09-12T14:56:01.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.036+0000 7f74c4ff9640 1 -- 172.21.15.131:0/136863672 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f74b0003b70 con 0x7f74c80ab730 2023-09-12T14:56:01.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.036+0000 7f74c4ff9640 1 --2- 172.21.15.131:0/136863672 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f74a8065b20 0x7f74a8067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:01.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.036+0000 7f74c77fe640 1 --2- 172.21.15.131:0/136863672 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f74a8065b20 0x7f74a8067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:01.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.036+0000 7f74c4ff9640 1 -- 172.21.15.131:0/136863672 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f74b0082a60 con 0x7f74c80ab730 2023-09-12T14:56:01.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.036+0000 7f74c77fe640 1 --2- 172.21.15.131:0/136863672 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f74a8065b20 0x7f74a8067fe0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f74c80b2510 tx=0x7f74b8009260 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:01.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.040+0000 7f74c4ff9640 1 -- 172.21.15.131:0/136863672 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f74b00503b0 con 0x7f74c80ab730 2023-09-12T14:56:01.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.188+0000 7f74cde3c640 1 -- 172.21.15.131:0/136863672 --> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f74c814e430 con 0x7f74a8065b20 2023-09-12T14:56:01.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.208+0000 7f74c4ff9640 1 -- 172.21.15.131:0/136863672 <== mgr.4200 v2:172.21.15.131:6832/2298103651 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+275756 (secure 0 0 0) 0x7f74c814e430 con 0x7f74a8065b20 2023-09-12T14:56:01.214 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:56:01.217 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-09-12T14:56:01.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.216+0000 7f74cde3c640 1 -- 172.21.15.131:0/136863672 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f74a8065b20 msgr2=0x7f74a8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:01.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.216+0000 7f74cde3c640 1 --2- 172.21.15.131:0/136863672 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f74a8065b20 0x7f74a8067fe0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f74c80b2510 tx=0x7f74b8009260 comp rx=0 tx=0).stop 2023-09-12T14:56:01.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.216+0000 7f74cde3c640 1 -- 172.21.15.131:0/136863672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74c80ab730 msgr2=0x7f74c8153120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:01.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.216+0000 7f74cde3c640 1 --2- 172.21.15.131:0/136863672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74c80ab730 0x7f74c8153120 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f74b000c3c0 tx=0x7f74b000c880 comp rx=0 tx=0).stop 2023-09-12T14:56:01.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.216+0000 7f74cde3c640 1 -- 172.21.15.131:0/136863672 shutdown_connections 2023-09-12T14:56:01.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.216+0000 7f74cde3c640 1 --2- 172.21.15.131:0/136863672 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f74a8065b20 0x7f74a8067fe0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:01.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.216+0000 7f74cde3c640 1 --2- 172.21.15.131:0/136863672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74c8158180 0x7f74c8148420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:01.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.216+0000 7f74cde3c640 1 --2- 172.21.15.131:0/136863672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74c80ab730 0x7f74c8153120 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:01.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.216+0000 7f74cde3c640 1 --2- 172.21.15.131:0/136863672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74c806b050 0x7f74c814e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:01.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.216+0000 7f74cde3c640 1 -- 172.21.15.131:0/136863672 >> 172.21.15.131:0/136863672 conn(0x7f74c8069840 msgr2=0x7f74c81593b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:01.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.216+0000 7f74cde3c640 1 -- 172.21.15.131:0/136863672 shutdown_connections 2023-09-12T14:56:01.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.216+0000 7f74cde3c640 1 -- 172.21.15.131:0/136863672 wait complete. 2023-09-12T14:56:01.235 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-09-12T14:55:59.595951+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219924,"kb_used_data":3696,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754796,"statfs":{"total":773094113280,"available":772868911104,"internally_reserved":0,"allocated":3784704,"data_stored":2159686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.101560"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.506038+0000","last_change":"2023-09-12T14:55:55.506038+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.506038+0000","last_undegraded":"2023-09-12T14:55:55.506038+0000","last_fullsized":"2023-09-12T14:55:55.506038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.501947+0000","last_change":"2023-09-12T14:55:55.501947+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.501947+0000","last_undegraded":"2023-09-12T14:55:55.501947+0000","last_fullsized":"2023-09-12T14:55:55.501947+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.503426+0000","last_change":"2023-09-12T14:55:55.503426+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.503426+0000","last_undegraded":"2023-09-12T14:55:55.503426+0000","last_fullsized":"2023-09-12T14:55:55.503426+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.506221+0000","last_change":"2023-09-12T14:55:55.506221+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.506221+0000","last_undegraded":"2023-09-12T14:55:55.506221+0000","last_fullsized":"2023-09-12T14:55:55.506221+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.505165+0000","last_change":"2023-09-12T14:55:55.505165+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.505165+0000","last_undegraded":"2023-09-12T14:55:55.505165+0000","last_fullsized":"2023-09-12T14:55:55.505165+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.499803+0000","last_change":"2023-09-12T14:55:55.499803+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.499803+0000","last_undegraded":"2023-09-12T14:55:55.499803+0000","last_fullsized":"2023-09-12T14:55:55.499803+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.504192+0000","last_change":"2023-09-12T14:55:55.504192+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.504192+0000","last_undegraded":"2023-09-12T14:55:55.504192+0000","last_fullsized":"2023-09-12T14:55:55.504192+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.499816+0000","last_change":"2023-09-12T14:55:55.499816+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.499816+0000","last_undegraded":"2023-09-12T14:55:55.499816+0000","last_fullsized":"2023-09-12T14:55:55.499816+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.504086+0000","last_change":"2023-09-12T14:55:55.504086+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.504086+0000","last_undegraded":"2023-09-12T14:55:55.504086+0000","last_fullsized":"2023-09-12T14:55:55.504086+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.501264+0000","last_change":"2023-09-12T14:55:55.501264+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.501264+0000","last_undegraded":"2023-09-12T14:55:55.501264+0000","last_fullsized":"2023-09-12T14:55:55.501264+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.500589+0000","last_change":"2023-09-12T14:55:55.500589+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.500589+0000","last_undegraded":"2023-09-12T14:55:55.500589+0000","last_fullsized":"2023-09-12T14:55:55.500589+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.503522+0000","last_change":"2023-09-12T14:55:55.503522+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.503522+0000","last_undegraded":"2023-09-12T14:55:55.503522+0000","last_fullsized":"2023-09-12T14:55:55.503522+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.502591+0000","last_change":"2023-09-12T14:55:55.502591+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.502591+0000","last_undegraded":"2023-09-12T14:55:55.502591+0000","last_fullsized":"2023-09-12T14:55:55.502591+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.503708+0000","last_change":"2023-09-12T14:55:55.503708+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.503708+0000","last_undegraded":"2023-09-12T14:55:55.503708+0000","last_fullsized":"2023-09-12T14:55:55.503708+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.506010+0000","last_change":"2023-09-12T14:55:55.506010+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.506010+0000","last_undegraded":"2023-09-12T14:55:55.506010+0000","last_fullsized":"2023-09-12T14:55:55.506010+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.503103+0000","last_change":"2023-09-12T14:55:55.503103+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.503103+0000","last_undegraded":"2023-09-12T14:55:55.503103+0000","last_fullsized":"2023-09-12T14:55:55.503103+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.503457+0000","last_change":"2023-09-12T14:55:55.503457+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.503457+0000","last_undegraded":"2023-09-12T14:55:55.503457+0000","last_fullsized":"2023-09-12T14:55:55.503457+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.500630+0000","last_change":"2023-09-12T14:55:55.500630+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.500630+0000","last_undegraded":"2023-09-12T14:55:55.500630+0000","last_fullsized":"2023-09-12T14:55:55.500630+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.499990+0000","last_change":"2023-09-12T14:55:55.499990+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.499990+0000","last_undegraded":"2023-09-12T14:55:55.499990+0000","last_fullsized":"2023-09-12T14:55:55.499990+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.504226+0000","last_change":"2023-09-12T14:55:55.504226+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.504226+0000","last_undegraded":"2023-09-12T14:55:55.504226+0000","last_fullsized":"2023-09-12T14:55:55.504226+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.500759+0000","last_change":"2023-09-12T14:55:55.500759+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.500759+0000","last_undegraded":"2023-09-12T14:55:55.500759+0000","last_fullsized":"2023-09-12T14:55:55.500759+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.506589+0000","last_change":"2023-09-12T14:55:55.506589+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.506589+0000","last_undegraded":"2023-09-12T14:55:55.506589+0000","last_fullsized":"2023-09-12T14:55:55.506589+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.504932+0000","last_change":"2023-09-12T14:55:55.504932+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.504932+0000","last_undegraded":"2023-09-12T14:55:55.504932+0000","last_fullsized":"2023-09-12T14:55:55.504932+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.499587+0000","last_change":"2023-09-12T14:55:55.499587+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.499587+0000","last_undegraded":"2023-09-12T14:55:55.499587+0000","last_fullsized":"2023-09-12T14:55:55.499587+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.502791+0000","last_change":"2023-09-12T14:55:55.502791+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.502791+0000","last_undegraded":"2023-09-12T14:55:55.502791+0000","last_fullsized":"2023-09-12T14:55:55.502791+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.499996+0000","last_change":"2023-09-12T14:55:55.499996+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.499996+0000","last_undegraded":"2023-09-12T14:55:55.499996+0000","last_fullsized":"2023-09-12T14:55:55.499996+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.503433+0000","last_change":"2023-09-12T14:55:55.503433+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.503433+0000","last_undegraded":"2023-09-12T14:55:55.503433+0000","last_fullsized":"2023-09-12T14:55:55.503433+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.499833+0000","last_change":"2023-09-12T14:55:55.499833+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.499833+0000","last_undegraded":"2023-09-12T14:55:55.499833+0000","last_fullsized":"2023-09-12T14:55:55.499833+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.503026+0000","last_change":"2023-09-12T14:55:55.503026+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.503026+0000","last_undegraded":"2023-09-12T14:55:55.503026+0000","last_fullsized":"2023-09-12T14:55:55.503026+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.503582+0000","last_change":"2023-09-12T14:55:55.503582+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.503582+0000","last_undegraded":"2023-09-12T14:55:55.503582+0000","last_fullsized":"2023-09-12T14:55:55.503582+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.504113+0000","last_change":"2023-09-12T14:55:55.504113+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.504113+0000","last_undegraded":"2023-09-12T14:55:55.504113+0000","last_fullsized":"2023-09-12T14:55:55.504113+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-12T14:55:56.493444+0000","last_change":"2023-09-12T14:55:56.493444+0000","last_active":"2023-09-12T14:55:56.493444+0000","last_peered":"2023-09-12T14:55:56.493444+0000","last_clean":"2023-09-12T14:55:56.493444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:56.493444+0000","last_undegraded":"2023-09-12T14:55:56.493444+0000","last_fullsized":"2023-09-12T14:55:56.493444+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-09-12T14:55:55.506309+0000","last_change":"2023-09-12T14:55:55.506309+0000","last_active":"2023-09-12T14:55:55.488872+0000","last_peered":"2023-09-12T14:55:55.488872+0000","last_clean":"2023-09-12T14:55:55.488872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-12T14:55:55.506309+0000","last_undegraded":"2023-09-12T14:55:55.506309+0000","last_fullsized":"2023-09-12T14:55:55.506309+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":173,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-09-12T14:55:51.718431+0000","last_change":"2023-09-12T14:54:46.965446+0000","last_active":"2023-09-12T14:55:51.718431+0000","last_peered":"2023-09-12T14:55:51.718431+0000","last_clean":"2023-09-12T14:55:51.718431+0000","last_became_active":"2023-09-12T14:54:46.963236+0000","last_became_peered":"2023-09-12T14:54:46.963236+0000","last_unstale":"2023-09-12T14:55:51.718431+0000","last_undegraded":"2023-09-12T14:55:51.718431+0000","last_fullsized":"2023-09-12T14:55:51.718431+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:45.804232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:45.804232+0000","last_clean_scrub_stamp":"2023-09-12T14:54:45.804232+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:24:45.670513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":917504,"data_stored":712137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":1.244},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":1.351}]},{"osd":1,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":1.2949999999999999}]},{"osd":2,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.073}]},{"osd":3,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.115},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.94399999999999995}]},{"osd":4,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.99299999999999999}]},{"osd":5,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.159}]},{"osd":6,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.04},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.2050000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":323584,"data_stored":121769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.65300000000000002}]},{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.84399999999999997}]},{"osd":2,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":1.0800000000000001}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.91300000000000003}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":1.133}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":1.113},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":1.1910000000000001}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":1.256}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":323584,"data_stored":121769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.94199999999999995}]},{"osd":1,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.64400000000000002}]},{"osd":2,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.56999999999999995}]},{"osd":3,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.753}]},{"osd":4,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.0900000000000001}]},{"osd":6,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.516,"5min":2.516,"15min":2.516},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.85199999999999998}]},{"osd":7,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.135},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":1}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":323584,"data_stored":121769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.81299999999999994}]},{"osd":1,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.86799999999999999}]},{"osd":2,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":0.94199999999999995}]},{"osd":3,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":1.1659999999999999}]},{"osd":5,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":1.044}]},{"osd":6,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":1.069},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":1.093}]},{"osd":7,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":1.222}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.31900000000000001}]},{"osd":1,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.35999999999999999}]},{"osd":2,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.30199999999999999}]},{"osd":4,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.76400000000000001}]},{"osd":5,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.64600000000000002}]},{"osd":6,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.61599999999999999}]},{"osd":7,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":0.626},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.81100000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":323584,"data_stored":121769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.47899999999999998}]},{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.441}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.41899999999999998}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.56399999999999995}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.60899999999999999}]},{"osd":6,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.1989999999999998,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":0.69099999999999995}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.63500000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":323584,"data_stored":121769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.627},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.78800000000000003}]},{"osd":2,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.499},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.85499999999999998}]},{"osd":3,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.755},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.54400000000000004}]},{"osd":4,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.94099999999999995}]},{"osd":5,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.879},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.95799999999999996}]},{"osd":6,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":1.105},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.91800000000000004}]},{"osd":7,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.0589999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":913408,"data_stored":711118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.66200000000000003}]},{"osd":2,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.76800000000000002}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":1.1519999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":1.127}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.054}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":1.091},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":0.94499999999999995}]},{"osd":6,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.1719999999999999}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.196}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-12T14:56:01.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2023-09-12T14:56:01.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2023-09-12T14:56:01.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2023-09-12T14:56:01.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2023-09-12T14:56:01.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2023-09-12T14:56:01.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2023-09-12T14:56:01.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.246 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2023-09-12T14:56:01.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.247 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2023-09-12T14:56:01.247 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2023-09-12T14:56:01.248 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2023-09-12T14:56:01.249 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.249 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2023-09-12T14:56:01.250 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.250 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2023-09-12T14:56:01.250 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2023-09-12T14:56:01.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2023-09-12T14:56:01.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2023-09-12T14:56:01.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2023-09-12T14:56:01.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.254 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2023-09-12T14:56:01.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.254 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2023-09-12T14:56:01.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2023-09-12T14:56:01.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2023-09-12T14:56:01.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2023-09-12T14:56:01.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2023-09-12T14:56:01.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2023-09-12T14:56:01.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2023-09-12T14:56:01.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2023-09-12T14:56:01.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2023-09-12T14:56:01.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2023-09-12T14:56:01.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2023-09-12T14:56:01.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2023-09-12T14:56:01.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2023-09-12T14:56:01.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2023-09-12T14:56:01.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2023-09-12T14:56:01.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2023-09-12T14:56:01.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1f is not active+clean 2023-09-12T14:56:01.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.506038+0000', 'last_change': '2023-09-12T14:55:55.506038+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.506038+0000', 'last_undegraded': '2023-09-12T14:55:55.506038+0000', 'last_fullsized': '2023-09-12T14:55:55.506038+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-09-12T14:56:01.268 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2023-09-12T14:56:01.268 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.268 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1e is not active+clean 2023-09-12T14:56:01.269 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.501947+0000', 'last_change': '2023-09-12T14:55:55.501947+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.501947+0000', 'last_undegraded': '2023-09-12T14:55:55.501947+0000', 'last_fullsized': '2023-09-12T14:55:55.501947+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-09-12T14:56:01.269 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2023-09-12T14:56:01.269 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.270 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1d is not active+clean 2023-09-12T14:56:01.270 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.503426+0000', 'last_change': '2023-09-12T14:55:55.503426+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.503426+0000', 'last_undegraded': '2023-09-12T14:55:55.503426+0000', 'last_fullsized': '2023-09-12T14:55:55.503426+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-09-12T14:56:01.271 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2023-09-12T14:56:01.272 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.273 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1c is not active+clean 2023-09-12T14:56:01.273 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.506221+0000', 'last_change': '2023-09-12T14:55:55.506221+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.506221+0000', 'last_undegraded': '2023-09-12T14:55:55.506221+0000', 'last_fullsized': '2023-09-12T14:55:55.506221+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-09-12T14:56:01.274 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2023-09-12T14:56:01.274 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.275 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1b is not active+clean 2023-09-12T14:56:01.275 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.505165+0000', 'last_change': '2023-09-12T14:55:55.505165+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.505165+0000', 'last_undegraded': '2023-09-12T14:55:55.505165+0000', 'last_fullsized': '2023-09-12T14:55:55.505165+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-09-12T14:56:01.275 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2023-09-12T14:56:01.276 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.276 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1a is not active+clean 2023-09-12T14:56:01.276 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.499803+0000', 'last_change': '2023-09-12T14:55:55.499803+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.499803+0000', 'last_undegraded': '2023-09-12T14:55:55.499803+0000', 'last_fullsized': '2023-09-12T14:55:55.499803+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-09-12T14:56:01.277 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2023-09-12T14:56:01.277 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.278 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.19 is not active+clean 2023-09-12T14:56:01.278 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.504192+0000', 'last_change': '2023-09-12T14:55:55.504192+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.504192+0000', 'last_undegraded': '2023-09-12T14:55:55.504192+0000', 'last_fullsized': '2023-09-12T14:55:55.504192+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-09-12T14:56:01.278 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2023-09-12T14:56:01.279 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.279 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.18 is not active+clean 2023-09-12T14:56:01.280 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.499816+0000', 'last_change': '2023-09-12T14:55:55.499816+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.499816+0000', 'last_undegraded': '2023-09-12T14:55:55.499816+0000', 'last_fullsized': '2023-09-12T14:55:55.499816+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-09-12T14:56:01.280 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2023-09-12T14:56:01.280 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.281 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.17 is not active+clean 2023-09-12T14:56:01.281 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.504086+0000', 'last_change': '2023-09-12T14:55:55.504086+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.504086+0000', 'last_undegraded': '2023-09-12T14:55:55.504086+0000', 'last_fullsized': '2023-09-12T14:55:55.504086+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-09-12T14:56:01.282 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2023-09-12T14:56:01.282 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.282 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.16 is not active+clean 2023-09-12T14:56:01.283 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.501264+0000', 'last_change': '2023-09-12T14:55:55.501264+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.501264+0000', 'last_undegraded': '2023-09-12T14:55:55.501264+0000', 'last_fullsized': '2023-09-12T14:55:55.501264+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-09-12T14:56:01.283 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2023-09-12T14:56:01.283 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.284 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.15 is not active+clean 2023-09-12T14:56:01.284 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.500589+0000', 'last_change': '2023-09-12T14:55:55.500589+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.500589+0000', 'last_undegraded': '2023-09-12T14:55:55.500589+0000', 'last_fullsized': '2023-09-12T14:55:55.500589+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-09-12T14:56:01.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2023-09-12T14:56:01.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.14 is not active+clean 2023-09-12T14:56:01.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.503522+0000', 'last_change': '2023-09-12T14:55:55.503522+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.503522+0000', 'last_undegraded': '2023-09-12T14:55:55.503522+0000', 'last_fullsized': '2023-09-12T14:55:55.503522+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-09-12T14:56:01.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2023-09-12T14:56:01.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.13 is not active+clean 2023-09-12T14:56:01.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.502591+0000', 'last_change': '2023-09-12T14:55:55.502591+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.502591+0000', 'last_undegraded': '2023-09-12T14:55:55.502591+0000', 'last_fullsized': '2023-09-12T14:55:55.502591+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-09-12T14:56:01.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2023-09-12T14:56:01.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.289 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.12 is not active+clean 2023-09-12T14:56:01.289 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.503708+0000', 'last_change': '2023-09-12T14:55:55.503708+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.503708+0000', 'last_undegraded': '2023-09-12T14:55:55.503708+0000', 'last_fullsized': '2023-09-12T14:55:55.503708+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-09-12T14:56:01.289 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2023-09-12T14:56:01.290 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.290 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.11 is not active+clean 2023-09-12T14:56:01.291 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.506010+0000', 'last_change': '2023-09-12T14:55:55.506010+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.506010+0000', 'last_undegraded': '2023-09-12T14:55:55.506010+0000', 'last_fullsized': '2023-09-12T14:55:55.506010+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-09-12T14:56:01.291 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2023-09-12T14:56:01.291 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.292 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.10 is not active+clean 2023-09-12T14:56:01.292 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.503103+0000', 'last_change': '2023-09-12T14:55:55.503103+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.503103+0000', 'last_undegraded': '2023-09-12T14:55:55.503103+0000', 'last_fullsized': '2023-09-12T14:55:55.503103+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-09-12T14:56:01.293 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2023-09-12T14:56:01.293 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.293 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.f is not active+clean 2023-09-12T14:56:01.294 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.503457+0000', 'last_change': '2023-09-12T14:55:55.503457+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.503457+0000', 'last_undegraded': '2023-09-12T14:55:55.503457+0000', 'last_fullsized': '2023-09-12T14:55:55.503457+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-09-12T14:56:01.294 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2023-09-12T14:56:01.294 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.295 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.e is not active+clean 2023-09-12T14:56:01.295 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.500630+0000', 'last_change': '2023-09-12T14:55:55.500630+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.500630+0000', 'last_undegraded': '2023-09-12T14:55:55.500630+0000', 'last_fullsized': '2023-09-12T14:55:55.500630+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-09-12T14:56:01.296 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2023-09-12T14:56:01.296 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.296 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.d is not active+clean 2023-09-12T14:56:01.297 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.499990+0000', 'last_change': '2023-09-12T14:55:55.499990+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.499990+0000', 'last_undegraded': '2023-09-12T14:55:55.499990+0000', 'last_fullsized': '2023-09-12T14:55:55.499990+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-09-12T14:56:01.297 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2023-09-12T14:56:01.298 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.298 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.c is not active+clean 2023-09-12T14:56:01.298 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.504226+0000', 'last_change': '2023-09-12T14:55:55.504226+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.504226+0000', 'last_undegraded': '2023-09-12T14:55:55.504226+0000', 'last_fullsized': '2023-09-12T14:55:55.504226+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-09-12T14:56:01.299 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2023-09-12T14:56:01.299 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.299 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.b is not active+clean 2023-09-12T14:56:01.300 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.500759+0000', 'last_change': '2023-09-12T14:55:55.500759+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.500759+0000', 'last_undegraded': '2023-09-12T14:55:55.500759+0000', 'last_fullsized': '2023-09-12T14:55:55.500759+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-09-12T14:56:01.300 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2023-09-12T14:56:01.301 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.301 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.a is not active+clean 2023-09-12T14:56:01.301 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.506589+0000', 'last_change': '2023-09-12T14:55:55.506589+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.506589+0000', 'last_undegraded': '2023-09-12T14:55:55.506589+0000', 'last_fullsized': '2023-09-12T14:55:55.506589+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-09-12T14:56:01.302 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2023-09-12T14:56:01.302 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.302 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.9 is not active+clean 2023-09-12T14:56:01.303 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.504932+0000', 'last_change': '2023-09-12T14:55:55.504932+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.504932+0000', 'last_undegraded': '2023-09-12T14:55:55.504932+0000', 'last_fullsized': '2023-09-12T14:55:55.504932+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-09-12T14:56:01.303 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2023-09-12T14:56:01.304 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.304 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.8 is not active+clean 2023-09-12T14:56:01.304 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.499587+0000', 'last_change': '2023-09-12T14:55:55.499587+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.499587+0000', 'last_undegraded': '2023-09-12T14:55:55.499587+0000', 'last_fullsized': '2023-09-12T14:55:55.499587+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-09-12T14:56:01.305 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2023-09-12T14:56:01.305 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.305 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.7 is not active+clean 2023-09-12T14:56:01.306 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.502791+0000', 'last_change': '2023-09-12T14:55:55.502791+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.502791+0000', 'last_undegraded': '2023-09-12T14:55:55.502791+0000', 'last_fullsized': '2023-09-12T14:55:55.502791+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-09-12T14:56:01.306 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2023-09-12T14:56:01.307 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.307 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.6 is not active+clean 2023-09-12T14:56:01.307 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.499996+0000', 'last_change': '2023-09-12T14:55:55.499996+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.499996+0000', 'last_undegraded': '2023-09-12T14:55:55.499996+0000', 'last_fullsized': '2023-09-12T14:55:55.499996+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-09-12T14:56:01.308 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2023-09-12T14:56:01.308 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.308 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.4 is not active+clean 2023-09-12T14:56:01.309 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.503433+0000', 'last_change': '2023-09-12T14:55:55.503433+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.503433+0000', 'last_undegraded': '2023-09-12T14:55:55.503433+0000', 'last_fullsized': '2023-09-12T14:55:55.503433+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-09-12T14:56:01.309 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2023-09-12T14:56:01.310 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.310 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3 is not active+clean 2023-09-12T14:56:01.310 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.499833+0000', 'last_change': '2023-09-12T14:55:55.499833+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.499833+0000', 'last_undegraded': '2023-09-12T14:55:55.499833+0000', 'last_fullsized': '2023-09-12T14:55:55.499833+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-09-12T14:56:01.311 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2023-09-12T14:56:01.311 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2 is not active+clean 2023-09-12T14:56:01.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.503026+0000', 'last_change': '2023-09-12T14:55:55.503026+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.503026+0000', 'last_undegraded': '2023-09-12T14:55:55.503026+0000', 'last_fullsized': '2023-09-12T14:55:55.503026+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-09-12T14:56:01.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2023-09-12T14:56:01.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1 is not active+clean 2023-09-12T14:56:01.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.503582+0000', 'last_change': '2023-09-12T14:55:55.503582+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.503582+0000', 'last_undegraded': '2023-09-12T14:55:55.503582+0000', 'last_fullsized': '2023-09-12T14:55:55.503582+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-09-12T14:56:01.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2023-09-12T14:56:01.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.0 is not active+clean 2023-09-12T14:56:01.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.504113+0000', 'last_change': '2023-09-12T14:55:55.504113+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.504113+0000', 'last_undegraded': '2023-09-12T14:55:55.504113+0000', 'last_fullsized': '2023-09-12T14:55:55.504113+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-09-12T14:56:01.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2023-09-12T14:56:01.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-12T14:55:56.493444+0000', 'last_change': '2023-09-12T14:55:56.493444+0000', 'last_active': '2023-09-12T14:55:56.493444+0000', 'last_peered': '2023-09-12T14:55:56.493444+0000', 'last_clean': '2023-09-12T14:55:56.493444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:56.493444+0000', 'last_undegraded': '2023-09-12T14:55:56.493444+0000', 'last_fullsized': '2023-09-12T14:55:56.493444+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:56.493444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-12T14:56:01.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.5 is not active+clean 2023-09-12T14:56:01.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-09-12T14:55:55.506309+0000', 'last_change': '2023-09-12T14:55:55.506309+0000', 'last_active': '2023-09-12T14:55:55.488872+0000', 'last_peered': '2023-09-12T14:55:55.488872+0000', 'last_clean': '2023-09-12T14:55:55.488872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-12T14:55:55.506309+0000', 'last_undegraded': '2023-09-12T14:55:55.506309+0000', 'last_fullsized': '2023-09-12T14:55:55.506309+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'last_clean_scrub_stamp': '2023-09-12T14:55:55.488872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-09-12T14:56:01.318 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-09-12T14:56:01.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.440+0000 7f5a86a00640 1 -- 172.21.15.131:0/744532092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a680047d0 msgr2=0x7f5a68004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:01.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.440+0000 7f5a86a00640 1 --2- 172.21.15.131:0/744532092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a680047d0 0x7f5a68004c50 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400c260 tx=0x7f5a7401f740 comp rx=0 tx=0).stop 2023-09-12T14:56:01.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.440+0000 7f5a86a00640 1 -- 172.21.15.131:0/744532092 shutdown_connections 2023-09-12T14:56:01.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.440+0000 7f5a86a00640 1 --2- 172.21.15.131:0/744532092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a680047d0 0x7f5a68004c50 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:01.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.440+0000 7f5a86a00640 1 -- 172.21.15.131:0/744532092 >> 172.21.15.131:0/744532092 conn(0x7f5a80069840 msgr2=0x7f5a800cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:01.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.440+0000 7f5a86a00640 1 -- 172.21.15.131:0/744532092 shutdown_connections 2023-09-12T14:56:01.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.440+0000 7f5a86a00640 1 -- 172.21.15.131:0/744532092 wait complete. 2023-09-12T14:56:01.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.444+0000 7f5a86a00640 1 Processor -- start 2023-09-12T14:56:01.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.444+0000 7f5a86a00640 1 -- start start 2023-09-12T14:56:01.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.444+0000 7f5a86a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a680047d0 0x7f5a8014d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:01.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.444+0000 7f5a86a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a8014b620 0x7f5a80141400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:01.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.444+0000 7f5a86a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a80143980 0x7f5a80143e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:01.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.444+0000 7f5a86a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a801434f0 con 0x7f5a80143980 2023-09-12T14:56:01.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.444+0000 7f5a86a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8014ddd0 con 0x7f5a8014b620 2023-09-12T14:56:01.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.444+0000 7f5a86a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a80142ea0 con 0x7f5a680047d0 2023-09-12T14:56:01.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.444+0000 7f5a84f9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a80143980 0x7f5a80143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:01.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.444+0000 7f5a84f9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a80143980 0x7f5a80143e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37408/0 (socket says 172.21.15.131:37408) 2023-09-12T14:56:01.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.444+0000 7f5a84f9d640 1 -- 172.21.15.131:0/2133280528 learned_addr learned my addr 172.21.15.131:0/2133280528 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:01.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.444+0000 7f5a7f7fe640 1 --2- 172.21.15.131:0/2133280528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a8014b620 0x7f5a80141400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:01.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.444+0000 7f5a7ffff640 1 --2- 172.21.15.131:0/2133280528 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a680047d0 0x7f5a8014d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:01.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.444+0000 7f5a84f9d640 1 -- 172.21.15.131:0/2133280528 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a680047d0 msgr2=0x7f5a8014d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:01.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.444+0000 7f5a84f9d640 1 --2- 172.21.15.131:0/2133280528 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a680047d0 0x7f5a8014d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:01.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.444+0000 7f5a84f9d640 1 -- 172.21.15.131:0/2133280528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a8014b620 msgr2=0x7f5a80141400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:01.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.444+0000 7f5a84f9d640 1 --2- 172.21.15.131:0/2133280528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a8014b620 0x7f5a80141400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:01.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.444+0000 7f5a84f9d640 1 -- 172.21.15.131:0/2133280528 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a800cbe00 con 0x7f5a80143980 2023-09-12T14:56:01.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.444+0000 7f5a7ffff640 1 --2- 172.21.15.131:0/2133280528 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a680047d0 0x7f5a8014d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:01.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.444+0000 7f5a84f9d640 1 --2- 172.21.15.131:0/2133280528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a80143980 0x7f5a80143e30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5a8006cf60 tx=0x7f5a7800d990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:01.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.444+0000 7f5a7cff9640 1 -- 172.21.15.131:0/2133280528 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a78003d90 con 0x7f5a80143980 2023-09-12T14:56:01.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.444+0000 7f5a7cff9640 1 -- 172.21.15.131:0/2133280528 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5a78006070 con 0x7f5a80143980 2023-09-12T14:56:01.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.444+0000 7f5a86a00640 1 -- 172.21.15.131:0/2133280528 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a80143180 con 0x7f5a80143980 2023-09-12T14:56:01.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.444+0000 7f5a7cff9640 1 -- 172.21.15.131:0/2133280528 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a78006460 con 0x7f5a80143980 2023-09-12T14:56:01.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.444+0000 7f5a86a00640 1 -- 172.21.15.131:0/2133280528 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a80142b50 con 0x7f5a80143980 2023-09-12T14:56:01.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.444+0000 7f5a86a00640 1 -- 172.21.15.131:0/2133280528 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a80073580 con 0x7f5a80143980 2023-09-12T14:56:01.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.448+0000 7f5a7cff9640 1 -- 172.21.15.131:0/2133280528 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f5a780065c0 con 0x7f5a80143980 2023-09-12T14:56:01.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.448+0000 7f5a7cff9640 1 --2- 172.21.15.131:0/2133280528 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5a6c065b50 0x7f5a6c068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:01.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.448+0000 7f5a7cff9640 1 -- 172.21.15.131:0/2133280528 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a780834f0 con 0x7f5a80143980 2023-09-12T14:56:01.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.448+0000 7f5a7ffff640 1 --2- 172.21.15.131:0/2133280528 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5a6c065b50 0x7f5a6c068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:01.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.448+0000 7f5a7ffff640 1 --2- 172.21.15.131:0/2133280528 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5a6c065b50 0x7f5a6c068010 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5a70009b20 tx=0x7f5a700091e0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:01.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.452+0000 7f5a7cff9640 1 -- 172.21.15.131:0/2133280528 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f5a78050e40 con 0x7f5a80143980 2023-09-12T14:56:01.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.664+0000 7f5a86a00640 1 -- 172.21.15.131:0/2133280528 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f5a8006bac0 con 0x7f5a80143980 2023-09-12T14:56:01.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.664+0000 7f5a7cff9640 1 -- 172.21.15.131:0/2133280528 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1526 (secure 0 0 0) 0x7f5a78054460 con 0x7f5a80143980 2023-09-12T14:56:01.666 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:56:01.667 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":89,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":33,"num_osds":8,"num_up_osds":8,"osd_up_since":1694530484,"num_in_osds":8,"osd_in_since":1694530475,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":225202176,"bytes_avail":772868911104,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":13,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5268}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2023-09-12T14:55:51.594464+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-09-12T14:56:01.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.664+0000 7f5a86a00640 1 -- 172.21.15.131:0/2133280528 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5a6c065b50 msgr2=0x7f5a6c068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:01.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.664+0000 7f5a86a00640 1 --2- 172.21.15.131:0/2133280528 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5a6c065b50 0x7f5a6c068010 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5a70009b20 tx=0x7f5a700091e0 comp rx=0 tx=0).stop 2023-09-12T14:56:01.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.664+0000 7f5a86a00640 1 -- 172.21.15.131:0/2133280528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a80143980 msgr2=0x7f5a80143e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:01.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.664+0000 7f5a86a00640 1 --2- 172.21.15.131:0/2133280528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a80143980 0x7f5a80143e30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5a8006cf60 tx=0x7f5a7800d990 comp rx=0 tx=0).stop 2023-09-12T14:56:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.664+0000 7f5a86a00640 1 -- 172.21.15.131:0/2133280528 shutdown_connections 2023-09-12T14:56:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.664+0000 7f5a86a00640 1 --2- 172.21.15.131:0/2133280528 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5a6c065b50 0x7f5a6c068010 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.664+0000 7f5a86a00640 1 --2- 172.21.15.131:0/2133280528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a80143980 0x7f5a80143e30 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.664+0000 7f5a86a00640 1 --2- 172.21.15.131:0/2133280528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a8014b620 0x7f5a80141400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.664+0000 7f5a86a00640 1 --2- 172.21.15.131:0/2133280528 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a680047d0 0x7f5a8014d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:01.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.664+0000 7f5a86a00640 1 -- 172.21.15.131:0/2133280528 >> 172.21.15.131:0/2133280528 conn(0x7f5a80069840 msgr2=0x7f5a800b1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:01.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.664+0000 7f5a86a00640 1 -- 172.21.15.131:0/2133280528 shutdown_connections 2023-09-12T14:56:01.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.664+0000 7f5a86a00640 1 -- 172.21.15.131:0/2133280528 wait complete. 2023-09-12T14:56:01.686 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-09-12T14:56:01.686 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-12T14:56:01.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.800+0000 7f6592e2c640 1 -- 172.21.15.131:0/1182651235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f658c158d70 msgr2=0x7f658c06b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:01.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.800+0000 7f6592e2c640 1 --2- 172.21.15.131:0/1182651235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f658c158d70 0x7f658c06b430 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f658c070ff0 tx=0x7f658801c810 comp rx=0 tx=0).stop 2023-09-12T14:56:01.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.800+0000 7f6592e2c640 1 -- 172.21.15.131:0/1182651235 shutdown_connections 2023-09-12T14:56:01.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.800+0000 7f6592e2c640 1 --2- 172.21.15.131:0/1182651235 >> v2:172.21.15.187:3300/0 conn(0x7f658c0abae0 0x7f658c0c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:01.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.800+0000 7f6592e2c640 1 --2- 172.21.15.131:0/1182651235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f658c158d70 0x7f658c06b430 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:01.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.800+0000 7f6592e2c640 1 -- 172.21.15.131:0/1182651235 >> 172.21.15.131:0/1182651235 conn(0x7f658c069840 msgr2=0x7f658c0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:01.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.800+0000 7f6592e2c640 1 -- 172.21.15.131:0/1182651235 shutdown_connections 2023-09-12T14:56:01.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.800+0000 7f6592e2c640 1 -- 172.21.15.131:0/1182651235 wait complete. 2023-09-12T14:56:01.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f6592e2c640 1 Processor -- start 2023-09-12T14:56:01.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f6592e2c640 1 -- start start 2023-09-12T14:56:01.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f6592e2c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f658c0abae0 0x7f658c14d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:01.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f6592e2c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f658c0cc430 0x7f658c151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:01.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f6592e2c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f658c158d70 0x7f658c153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:01.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f6592e2c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658c14fb80 con 0x7f658c0abae0 2023-09-12T14:56:01.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f6592e2c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658c14fdd0 con 0x7f658c158d70 2023-09-12T14:56:01.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f6592e2c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f658c1500b0 con 0x7f658c0cc430 2023-09-12T14:56:01.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f6590bc8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f658c0abae0 0x7f658c14d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:01.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f6590bc8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f658c0abae0 0x7f658c14d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37432/0 (socket says 172.21.15.131:37432) 2023-09-12T14:56:01.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f6590bc8640 1 -- 172.21.15.131:0/4150864762 learned_addr learned my addr 172.21.15.131:0/4150864762 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:01.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f6583fff640 1 --2- 172.21.15.131:0/4150864762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f658c0cc430 0x7f658c151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:01.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f6590bc8640 1 -- 172.21.15.131:0/4150864762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f658c0cc430 msgr2=0x7f658c151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:01.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f6590bc8640 1 --2- 172.21.15.131:0/4150864762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f658c0cc430 0x7f658c151290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:01.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f65913c9640 1 --2- 172.21.15.131:0/4150864762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f658c158d70 0x7f658c153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:01.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f6590bc8640 1 -- 172.21.15.131:0/4150864762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f658c158d70 msgr2=0x7f658c153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:01.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f6590bc8640 1 --2- 172.21.15.131:0/4150864762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f658c158d70 0x7f658c153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:01.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f6590bc8640 1 -- 172.21.15.131:0/4150864762 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6588009560 con 0x7f658c0abae0 2023-09-12T14:56:01.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f6583fff640 1 --2- 172.21.15.131:0/4150864762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f658c0cc430 0x7f658c151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:01.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f65913c9640 1 --2- 172.21.15.131:0/4150864762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f658c158d70 0x7f658c153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:01.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f6590bc8640 1 --2- 172.21.15.131:0/4150864762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f658c0abae0 0x7f658c14d3e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f65880b0040 tx=0x7f65880040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:01.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f65817fa640 1 -- 172.21.15.131:0/4150864762 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65880bc070 con 0x7f658c0abae0 2023-09-12T14:56:01.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f65817fa640 1 -- 172.21.15.131:0/4150864762 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6588005030 con 0x7f658c0abae0 2023-09-12T14:56:01.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f6592e2c640 1 -- 172.21.15.131:0/4150864762 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f658c150390 con 0x7f658c0abae0 2023-09-12T14:56:01.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f65817fa640 1 -- 172.21.15.131:0/4150864762 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65880053e0 con 0x7f658c0abae0 2023-09-12T14:56:01.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f6592e2c640 1 -- 172.21.15.131:0/4150864762 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f658c14d920 con 0x7f658c0abae0 2023-09-12T14:56:01.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f6592e2c640 1 -- 172.21.15.131:0/4150864762 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f658c073580 con 0x7f658c0abae0 2023-09-12T14:56:01.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.804+0000 7f65817fa640 1 -- 172.21.15.131:0/4150864762 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f6588005540 con 0x7f658c0abae0 2023-09-12T14:56:01.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.808+0000 7f65817fa640 1 --2- 172.21.15.131:0/4150864762 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6578065b20 0x7f6578067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:01.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.808+0000 7f65817fa640 1 -- 172.21.15.131:0/4150864762 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65880aa070 con 0x7f658c0abae0 2023-09-12T14:56:01.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.808+0000 7f6583fff640 1 --2- 172.21.15.131:0/4150864762 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6578065b20 0x7f6578067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:01.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.808+0000 7f6583fff640 1 --2- 172.21.15.131:0/4150864762 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6578065b20 0x7f6578067fe0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f657c005fd0 tx=0x7f657c005d40 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:01.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.812+0000 7f65817fa640 1 -- 172.21.15.131:0/4150864762 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f65880eb850 con 0x7f658c0abae0 2023-09-12T14:56:01.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.960+0000 7f6592e2c640 1 -- 172.21.15.131:0/4150864762 --> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f658c0cd4d0 con 0x7f6578065b20 2023-09-12T14:56:01.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.968+0000 7f65817fa640 1 -- 172.21.15.131:0/4150864762 <== mgr.4200 v2:172.21.15.131:6832/2298103651 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287773 (secure 0 0 0) 0x7f658c0cd4d0 con 0x7f6578065b20 2023-09-12T14:56:01.975 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:56:01.977 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-09-12T14:56:01.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.976+0000 7f6592e2c640 1 -- 172.21.15.131:0/4150864762 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6578065b20 msgr2=0x7f6578067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:01.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.976+0000 7f6592e2c640 1 --2- 172.21.15.131:0/4150864762 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6578065b20 0x7f6578067fe0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f657c005fd0 tx=0x7f657c005d40 comp rx=0 tx=0).stop 2023-09-12T14:56:01.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.976+0000 7f6592e2c640 1 -- 172.21.15.131:0/4150864762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f658c0abae0 msgr2=0x7f658c14d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:01.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.976+0000 7f6592e2c640 1 --2- 172.21.15.131:0/4150864762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f658c0abae0 0x7f658c14d3e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f65880b0040 tx=0x7f65880040c0 comp rx=0 tx=0).stop 2023-09-12T14:56:01.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.976+0000 7f6592e2c640 1 -- 172.21.15.131:0/4150864762 shutdown_connections 2023-09-12T14:56:01.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.976+0000 7f6592e2c640 1 --2- 172.21.15.131:0/4150864762 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6578065b20 0x7f6578067fe0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:01.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.976+0000 7f6592e2c640 1 --2- 172.21.15.131:0/4150864762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f658c0cc430 0x7f658c151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:01.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.976+0000 7f6592e2c640 1 --2- 172.21.15.131:0/4150864762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f658c158d70 0x7f658c153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:01.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.976+0000 7f6592e2c640 1 --2- 172.21.15.131:0/4150864762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f658c0abae0 0x7f658c14d3e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:01.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.976+0000 7f6592e2c640 1 -- 172.21.15.131:0/4150864762 >> 172.21.15.131:0/4150864762 conn(0x7f658c069840 msgr2=0x7f658c0ae690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:01.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.976+0000 7f6592e2c640 1 -- 172.21.15.131:0/4150864762 shutdown_connections 2023-09-12T14:56:01.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:01.976+0000 7f6592e2c640 1 -- 172.21.15.131:0/4150864762 wait complete. 2023-09-12T14:56:01.993 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2023-09-12T14:56:01.596362+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220312,"kb_used_data":4132,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754408,"statfs":{"total":773094113280,"available":772868513792,"internally_reserved":0,"allocated":4231168,"data_stored":2359961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":66,"apply_latency_ms":66,"commit_latency_ns":66000000,"apply_latency_ns":66000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001016"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:58.234088+0000","last_change":"2023-09-12T14:55:56.947127+0000","last_active":"2023-09-12T14:55:58.234088+0000","last_peered":"2023-09-12T14:55:58.234088+0000","last_clean":"2023-09-12T14:55:58.234088+0000","last_became_active":"2023-09-12T14:55:56.946228+0000","last_became_peered":"2023-09-12T14:55:56.946228+0000","last_unstale":"2023-09-12T14:55:58.234088+0000","last_undegraded":"2023-09-12T14:55:58.234088+0000","last_fullsized":"2023-09-12T14:55:58.234088+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:05:47.806674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.193417+0000","last_change":"2023-09-12T14:55:56.927280+0000","last_active":"2023-09-12T14:55:59.193417+0000","last_peered":"2023-09-12T14:55:59.193417+0000","last_clean":"2023-09-12T14:55:59.193417+0000","last_became_active":"2023-09-12T14:55:56.924706+0000","last_became_peered":"2023-09-12T14:55:56.924706+0000","last_unstale":"2023-09-12T14:55:59.193417+0000","last_undegraded":"2023-09-12T14:55:59.193417+0000","last_fullsized":"2023-09-12T14:55:59.193417+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:31:58.066578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.911466+0000","last_change":"2023-09-12T14:55:56.940406+0000","last_active":"2023-09-12T14:55:57.911466+0000","last_peered":"2023-09-12T14:55:57.911466+0000","last_clean":"2023-09-12T14:55:57.911466+0000","last_became_active":"2023-09-12T14:55:56.939600+0000","last_became_peered":"2023-09-12T14:55:56.939600+0000","last_unstale":"2023-09-12T14:55:57.911466+0000","last_undegraded":"2023-09-12T14:55:57.911466+0000","last_fullsized":"2023-09-12T14:55:57.911466+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:17:33.154086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.191800+0000","last_change":"2023-09-12T14:55:56.950303+0000","last_active":"2023-09-12T14:55:59.191800+0000","last_peered":"2023-09-12T14:55:59.191800+0000","last_clean":"2023-09-12T14:55:59.191800+0000","last_became_active":"2023-09-12T14:55:56.949704+0000","last_became_peered":"2023-09-12T14:55:56.949704+0000","last_unstale":"2023-09-12T14:55:59.191800+0000","last_undegraded":"2023-09-12T14:55:59.191800+0000","last_fullsized":"2023-09-12T14:55:59.191800+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:08:56.584164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189875+0000","last_change":"2023-09-12T14:55:56.920821+0000","last_active":"2023-09-12T14:55:59.189875+0000","last_peered":"2023-09-12T14:55:59.189875+0000","last_clean":"2023-09-12T14:55:59.189875+0000","last_became_active":"2023-09-12T14:55:56.920251+0000","last_became_peered":"2023-09-12T14:55:56.920251+0000","last_unstale":"2023-09-12T14:55:59.189875+0000","last_undegraded":"2023-09-12T14:55:59.189875+0000","last_fullsized":"2023-09-12T14:55:59.189875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:12:36.768826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189089+0000","last_change":"2023-09-12T14:55:56.939064+0000","last_active":"2023-09-12T14:55:59.189089+0000","last_peered":"2023-09-12T14:55:59.189089+0000","last_clean":"2023-09-12T14:55:59.189089+0000","last_became_active":"2023-09-12T14:55:56.938751+0000","last_became_peered":"2023-09-12T14:55:56.938751+0000","last_unstale":"2023-09-12T14:55:59.189089+0000","last_undegraded":"2023-09-12T14:55:59.189089+0000","last_fullsized":"2023-09-12T14:55:59.189089+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:17:33.154086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189848+0000","last_change":"2023-09-12T14:55:56.958364+0000","last_active":"2023-09-12T14:55:59.189848+0000","last_peered":"2023-09-12T14:55:59.189848+0000","last_clean":"2023-09-12T14:55:59.189848+0000","last_became_active":"2023-09-12T14:55:56.957011+0000","last_became_peered":"2023-09-12T14:55:56.957011+0000","last_unstale":"2023-09-12T14:55:59.189848+0000","last_undegraded":"2023-09-12T14:55:59.189848+0000","last_fullsized":"2023-09-12T14:55:59.189848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:53:01.273053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.188843+0000","last_change":"2023-09-12T14:55:56.940471+0000","last_active":"2023-09-12T14:55:59.188843+0000","last_peered":"2023-09-12T14:55:59.188843+0000","last_clean":"2023-09-12T14:55:59.188843+0000","last_became_active":"2023-09-12T14:55:56.939686+0000","last_became_peered":"2023-09-12T14:55:56.939686+0000","last_unstale":"2023-09-12T14:55:59.188843+0000","last_undegraded":"2023-09-12T14:55:59.188843+0000","last_fullsized":"2023-09-12T14:55:59.188843+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:10:13.906110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.910516+0000","last_change":"2023-09-12T14:55:56.951450+0000","last_active":"2023-09-12T14:55:57.910516+0000","last_peered":"2023-09-12T14:55:57.910516+0000","last_clean":"2023-09-12T14:55:57.910516+0000","last_became_active":"2023-09-12T14:55:56.950607+0000","last_became_peered":"2023-09-12T14:55:56.950607+0000","last_unstale":"2023-09-12T14:55:57.910516+0000","last_undegraded":"2023-09-12T14:55:57.910516+0000","last_fullsized":"2023-09-12T14:55:57.910516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:05:47.806674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.191986+0000","last_change":"2023-09-12T14:55:56.938513+0000","last_active":"2023-09-12T14:55:59.191986+0000","last_peered":"2023-09-12T14:55:59.191986+0000","last_clean":"2023-09-12T14:55:59.191986+0000","last_became_active":"2023-09-12T14:55:56.935930+0000","last_became_peered":"2023-09-12T14:55:56.935930+0000","last_unstale":"2023-09-12T14:55:59.191986+0000","last_undegraded":"2023-09-12T14:55:59.191986+0000","last_fullsized":"2023-09-12T14:55:59.191986+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:32:13.749664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189504+0000","last_change":"2023-09-12T14:55:56.955591+0000","last_active":"2023-09-12T14:55:59.189504+0000","last_peered":"2023-09-12T14:55:59.189504+0000","last_clean":"2023-09-12T14:55:59.189504+0000","last_became_active":"2023-09-12T14:55:56.954971+0000","last_became_peered":"2023-09-12T14:55:56.954971+0000","last_unstale":"2023-09-12T14:55:59.189504+0000","last_undegraded":"2023-09-12T14:55:59.189504+0000","last_fullsized":"2023-09-12T14:55:59.189504+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:37:00.305524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.911366+0000","last_change":"2023-09-12T14:55:56.937413+0000","last_active":"2023-09-12T14:55:57.911366+0000","last_peered":"2023-09-12T14:55:57.911366+0000","last_clean":"2023-09-12T14:55:57.911366+0000","last_became_active":"2023-09-12T14:55:56.936637+0000","last_became_peered":"2023-09-12T14:55:56.936637+0000","last_unstale":"2023-09-12T14:55:57.911366+0000","last_undegraded":"2023-09-12T14:55:57.911366+0000","last_fullsized":"2023-09-12T14:55:57.911366+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:35:17.945837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.190266+0000","last_change":"2023-09-12T14:55:56.939993+0000","last_active":"2023-09-12T14:55:59.190266+0000","last_peered":"2023-09-12T14:55:59.190266+0000","last_clean":"2023-09-12T14:55:59.190266+0000","last_became_active":"2023-09-12T14:55:56.939681+0000","last_became_peered":"2023-09-12T14:55:56.939681+0000","last_unstale":"2023-09-12T14:55:59.190266+0000","last_undegraded":"2023-09-12T14:55:59.190266+0000","last_fullsized":"2023-09-12T14:55:59.190266+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:31:40.972277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189873+0000","last_change":"2023-09-12T14:55:56.919959+0000","last_active":"2023-09-12T14:55:59.189873+0000","last_peered":"2023-09-12T14:55:59.189873+0000","last_clean":"2023-09-12T14:55:59.189873+0000","last_became_active":"2023-09-12T14:55:56.919251+0000","last_became_peered":"2023-09-12T14:55:56.919251+0000","last_unstale":"2023-09-12T14:55:59.189873+0000","last_undegraded":"2023-09-12T14:55:59.189873+0000","last_fullsized":"2023-09-12T14:55:59.189873+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:53:01.273053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.909815+0000","last_change":"2023-09-12T14:55:56.946665+0000","last_active":"2023-09-12T14:55:57.909815+0000","last_peered":"2023-09-12T14:55:57.909815+0000","last_clean":"2023-09-12T14:55:57.909815+0000","last_became_active":"2023-09-12T14:55:56.945745+0000","last_became_peered":"2023-09-12T14:55:56.945745+0000","last_unstale":"2023-09-12T14:55:57.909815+0000","last_undegraded":"2023-09-12T14:55:57.909815+0000","last_fullsized":"2023-09-12T14:55:57.909815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:37:00.305524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189744+0000","last_change":"2023-09-12T14:55:56.944586+0000","last_active":"2023-09-12T14:55:59.189744+0000","last_peered":"2023-09-12T14:55:59.189744+0000","last_clean":"2023-09-12T14:55:59.189744+0000","last_became_active":"2023-09-12T14:55:56.943452+0000","last_became_peered":"2023-09-12T14:55:56.943452+0000","last_unstale":"2023-09-12T14:55:59.189744+0000","last_undegraded":"2023-09-12T14:55:59.189744+0000","last_fullsized":"2023-09-12T14:55:59.189744+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:17:33.154086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189888+0000","last_change":"2023-09-12T14:55:56.957246+0000","last_active":"2023-09-12T14:55:59.189888+0000","last_peered":"2023-09-12T14:55:59.189888+0000","last_clean":"2023-09-12T14:55:59.189888+0000","last_became_active":"2023-09-12T14:55:56.956737+0000","last_became_peered":"2023-09-12T14:55:56.956737+0000","last_unstale":"2023-09-12T14:55:59.189888+0000","last_undegraded":"2023-09-12T14:55:59.189888+0000","last_fullsized":"2023-09-12T14:55:59.189888+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:54:34.731553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.193648+0000","last_change":"2023-09-12T14:55:56.944981+0000","last_active":"2023-09-12T14:55:59.193648+0000","last_peered":"2023-09-12T14:55:59.193648+0000","last_clean":"2023-09-12T14:55:59.193648+0000","last_became_active":"2023-09-12T14:55:56.944227+0000","last_became_peered":"2023-09-12T14:55:56.944227+0000","last_unstale":"2023-09-12T14:55:59.193648+0000","last_undegraded":"2023-09-12T14:55:59.193648+0000","last_fullsized":"2023-09-12T14:55:59.193648+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:21:38.490784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.191590+0000","last_change":"2023-09-12T14:55:56.942243+0000","last_active":"2023-09-12T14:55:59.191590+0000","last_peered":"2023-09-12T14:55:59.191590+0000","last_clean":"2023-09-12T14:55:59.191590+0000","last_became_active":"2023-09-12T14:55:56.941715+0000","last_became_peered":"2023-09-12T14:55:56.941715+0000","last_unstale":"2023-09-12T14:55:59.191590+0000","last_undegraded":"2023-09-12T14:55:59.191590+0000","last_fullsized":"2023-09-12T14:55:59.191590+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:47:13.451453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189830+0000","last_change":"2023-09-12T14:55:56.946050+0000","last_active":"2023-09-12T14:55:59.189830+0000","last_peered":"2023-09-12T14:55:59.189830+0000","last_clean":"2023-09-12T14:55:59.189830+0000","last_became_active":"2023-09-12T14:55:56.945621+0000","last_became_peered":"2023-09-12T14:55:56.945621+0000","last_unstale":"2023-09-12T14:55:59.189830+0000","last_undegraded":"2023-09-12T14:55:59.189830+0000","last_fullsized":"2023-09-12T14:55:59.189830+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:54:06.886063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.910461+0000","last_change":"2023-09-12T14:55:56.932342+0000","last_active":"2023-09-12T14:55:57.910461+0000","last_peered":"2023-09-12T14:55:57.910461+0000","last_clean":"2023-09-12T14:55:57.910461+0000","last_became_active":"2023-09-12T14:55:56.931596+0000","last_became_peered":"2023-09-12T14:55:56.931596+0000","last_unstale":"2023-09-12T14:55:57.910461+0000","last_undegraded":"2023-09-12T14:55:57.910461+0000","last_fullsized":"2023-09-12T14:55:57.910461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:24:53.210051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.909956+0000","last_change":"2023-09-12T14:55:56.937472+0000","last_active":"2023-09-12T14:55:57.909956+0000","last_peered":"2023-09-12T14:55:57.909956+0000","last_clean":"2023-09-12T14:55:57.909956+0000","last_became_active":"2023-09-12T14:55:56.936724+0000","last_became_peered":"2023-09-12T14:55:56.936724+0000","last_unstale":"2023-09-12T14:55:57.909956+0000","last_undegraded":"2023-09-12T14:55:57.909956+0000","last_fullsized":"2023-09-12T14:55:57.909956+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:24:05.843309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.903393+0000","last_change":"2023-09-12T14:55:56.941598+0000","last_active":"2023-09-12T14:55:57.903393+0000","last_peered":"2023-09-12T14:55:57.903393+0000","last_clean":"2023-09-12T14:55:57.903393+0000","last_became_active":"2023-09-12T14:55:56.941328+0000","last_became_peered":"2023-09-12T14:55:56.941328+0000","last_unstale":"2023-09-12T14:55:57.903393+0000","last_undegraded":"2023-09-12T14:55:57.903393+0000","last_fullsized":"2023-09-12T14:55:57.903393+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:05:47.806674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.909123+0000","last_change":"2023-09-12T14:55:56.920070+0000","last_active":"2023-09-12T14:55:57.909123+0000","last_peered":"2023-09-12T14:55:57.909123+0000","last_clean":"2023-09-12T14:55:57.909123+0000","last_became_active":"2023-09-12T14:55:56.919314+0000","last_became_peered":"2023-09-12T14:55:56.919314+0000","last_unstale":"2023-09-12T14:55:57.909123+0000","last_undegraded":"2023-09-12T14:55:57.909123+0000","last_fullsized":"2023-09-12T14:55:57.909123+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:32:13.749664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.192403+0000","last_change":"2023-09-12T14:55:56.941023+0000","last_active":"2023-09-12T14:55:59.192403+0000","last_peered":"2023-09-12T14:55:59.192403+0000","last_clean":"2023-09-12T14:55:59.192403+0000","last_became_active":"2023-09-12T14:55:56.940523+0000","last_became_peered":"2023-09-12T14:55:56.940523+0000","last_unstale":"2023-09-12T14:55:59.192403+0000","last_undegraded":"2023-09-12T14:55:59.192403+0000","last_fullsized":"2023-09-12T14:55:59.192403+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:35:17.945837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.193633+0000","last_change":"2023-09-12T14:55:56.954316+0000","last_active":"2023-09-12T14:55:59.193633+0000","last_peered":"2023-09-12T14:55:59.193633+0000","last_clean":"2023-09-12T14:55:59.193633+0000","last_became_active":"2023-09-12T14:55:56.953782+0000","last_became_peered":"2023-09-12T14:55:56.953782+0000","last_unstale":"2023-09-12T14:55:59.193633+0000","last_undegraded":"2023-09-12T14:55:59.193633+0000","last_fullsized":"2023-09-12T14:55:59.193633+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:26:56.144483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.192198+0000","last_change":"2023-09-12T14:55:56.950280+0000","last_active":"2023-09-12T14:55:59.192198+0000","last_peered":"2023-09-12T14:55:59.192198+0000","last_clean":"2023-09-12T14:55:59.192198+0000","last_became_active":"2023-09-12T14:55:56.949563+0000","last_became_peered":"2023-09-12T14:55:56.949563+0000","last_unstale":"2023-09-12T14:55:59.192198+0000","last_undegraded":"2023-09-12T14:55:59.192198+0000","last_fullsized":"2023-09-12T14:55:59.192198+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:26:20.168808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.911721+0000","last_change":"2023-09-12T14:55:56.936036+0000","last_active":"2023-09-12T14:55:57.911721+0000","last_peered":"2023-09-12T14:55:57.911721+0000","last_clean":"2023-09-12T14:55:57.911721+0000","last_became_active":"2023-09-12T14:55:56.935230+0000","last_became_peered":"2023-09-12T14:55:56.935230+0000","last_unstale":"2023-09-12T14:55:57.911721+0000","last_undegraded":"2023-09-12T14:55:57.911721+0000","last_fullsized":"2023-09-12T14:55:57.911721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:53:01.273053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.910903+0000","last_change":"2023-09-12T14:55:56.939673+0000","last_active":"2023-09-12T14:55:57.910903+0000","last_peered":"2023-09-12T14:55:57.910903+0000","last_clean":"2023-09-12T14:55:57.910903+0000","last_became_active":"2023-09-12T14:55:56.938856+0000","last_became_peered":"2023-09-12T14:55:56.938856+0000","last_unstale":"2023-09-12T14:55:57.910903+0000","last_undegraded":"2023-09-12T14:55:57.910903+0000","last_fullsized":"2023-09-12T14:55:57.910903+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:22:06.073591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189711+0000","last_change":"2023-09-12T14:55:56.959107+0000","last_active":"2023-09-12T14:55:59.189711+0000","last_peered":"2023-09-12T14:55:59.189711+0000","last_clean":"2023-09-12T14:55:59.189711+0000","last_became_active":"2023-09-12T14:55:56.957834+0000","last_became_peered":"2023-09-12T14:55:56.957834+0000","last_unstale":"2023-09-12T14:55:59.189711+0000","last_undegraded":"2023-09-12T14:55:59.189711+0000","last_fullsized":"2023-09-12T14:55:59.189711+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:35:17.945837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:58.233870+0000","last_change":"2023-09-12T14:55:56.942862+0000","last_active":"2023-09-12T14:55:58.233870+0000","last_peered":"2023-09-12T14:55:58.233870+0000","last_clean":"2023-09-12T14:55:58.233870+0000","last_became_active":"2023-09-12T14:55:56.941962+0000","last_became_peered":"2023-09-12T14:55:56.941962+0000","last_unstale":"2023-09-12T14:55:58.233870+0000","last_undegraded":"2023-09-12T14:55:58.233870+0000","last_fullsized":"2023-09-12T14:55:58.233870+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:50:12.068597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189026+0000","last_change":"2023-09-12T14:55:56.960407+0000","last_active":"2023-09-12T14:55:59.189026+0000","last_peered":"2023-09-12T14:55:59.189026+0000","last_clean":"2023-09-12T14:55:59.189026+0000","last_became_active":"2023-09-12T14:55:56.959786+0000","last_became_peered":"2023-09-12T14:55:56.959786+0000","last_unstale":"2023-09-12T14:55:59.189026+0000","last_undegraded":"2023-09-12T14:55:59.189026+0000","last_fullsized":"2023-09-12T14:55:59.189026+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:51:57.331781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.903349+0000","last_change":"2023-09-12T14:55:56.954445+0000","last_active":"2023-09-12T14:55:57.903349+0000","last_peered":"2023-09-12T14:55:57.903349+0000","last_clean":"2023-09-12T14:55:57.903349+0000","last_became_active":"2023-09-12T14:55:56.953823+0000","last_became_peered":"2023-09-12T14:55:56.953823+0000","last_unstale":"2023-09-12T14:55:57.903349+0000","last_undegraded":"2023-09-12T14:55:57.903349+0000","last_fullsized":"2023-09-12T14:55:57.903349+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:48:44.893608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.529461+0000","last_change":"2023-09-12T14:55:56.539868+0000","last_active":"2023-09-12T14:55:57.529461+0000","last_peered":"2023-09-12T14:55:57.529461+0000","last_clean":"2023-09-12T14:55:57.529461+0000","last_became_active":"2023-09-12T14:55:56.539227+0000","last_became_peered":"2023-09-12T14:55:56.539227+0000","last_unstale":"2023-09-12T14:55:57.529461+0000","last_undegraded":"2023-09-12T14:55:57.529461+0000","last_fullsized":"2023-09-12T14:55:57.529461+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:44:26.291246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:58.234133+0000","last_change":"2023-09-12T14:55:56.956878+0000","last_active":"2023-09-12T14:55:58.234133+0000","last_peered":"2023-09-12T14:55:58.234133+0000","last_clean":"2023-09-12T14:55:58.234133+0000","last_became_active":"2023-09-12T14:55:56.956210+0000","last_became_peered":"2023-09-12T14:55:56.956210+0000","last_unstale":"2023-09-12T14:55:58.234133+0000","last_undegraded":"2023-09-12T14:55:58.234133+0000","last_fullsized":"2023-09-12T14:55:58.234133+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:22:06.073591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.529479+0000","last_change":"2023-09-12T14:55:56.549272+0000","last_active":"2023-09-12T14:55:57.529479+0000","last_peered":"2023-09-12T14:55:57.529479+0000","last_clean":"2023-09-12T14:55:57.529479+0000","last_became_active":"2023-09-12T14:55:56.548890+0000","last_became_peered":"2023-09-12T14:55:56.548890+0000","last_unstale":"2023-09-12T14:55:57.529479+0000","last_undegraded":"2023-09-12T14:55:57.529479+0000","last_fullsized":"2023-09-12T14:55:57.529479+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:47:23.257700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.910356+0000","last_change":"2023-09-12T14:55:56.940012+0000","last_active":"2023-09-12T14:55:57.910356+0000","last_peered":"2023-09-12T14:55:57.910356+0000","last_clean":"2023-09-12T14:55:57.910356+0000","last_became_active":"2023-09-12T14:55:56.938537+0000","last_became_peered":"2023-09-12T14:55:56.938537+0000","last_unstale":"2023-09-12T14:55:57.910356+0000","last_undegraded":"2023-09-12T14:55:57.910356+0000","last_fullsized":"2023-09-12T14:55:57.910356+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:48:44.893608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.530119+0000","last_change":"2023-09-12T14:55:56.534734+0000","last_active":"2023-09-12T14:55:57.530119+0000","last_peered":"2023-09-12T14:55:57.530119+0000","last_clean":"2023-09-12T14:55:57.530119+0000","last_became_active":"2023-09-12T14:55:56.534334+0000","last_became_peered":"2023-09-12T14:55:56.534334+0000","last_unstale":"2023-09-12T14:55:57.530119+0000","last_undegraded":"2023-09-12T14:55:57.530119+0000","last_fullsized":"2023-09-12T14:55:57.530119+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:05:46.802102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.192370+0000","last_change":"2023-09-12T14:55:56.950640+0000","last_active":"2023-09-12T14:55:59.192370+0000","last_peered":"2023-09-12T14:55:59.192370+0000","last_clean":"2023-09-12T14:55:59.192370+0000","last_became_active":"2023-09-12T14:55:56.949846+0000","last_became_peered":"2023-09-12T14:55:56.949846+0000","last_unstale":"2023-09-12T14:55:59.192370+0000","last_undegraded":"2023-09-12T14:55:59.192370+0000","last_fullsized":"2023-09-12T14:55:59.192370+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:24:20.331640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.525109+0000","last_change":"2023-09-12T14:55:56.536652+0000","last_active":"2023-09-12T14:55:57.525109+0000","last_peered":"2023-09-12T14:55:57.525109+0000","last_clean":"2023-09-12T14:55:57.525109+0000","last_became_active":"2023-09-12T14:55:56.536281+0000","last_became_peered":"2023-09-12T14:55:56.536281+0000","last_unstale":"2023-09-12T14:55:57.525109+0000","last_undegraded":"2023-09-12T14:55:57.525109+0000","last_fullsized":"2023-09-12T14:55:57.525109+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:54:13.129205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189466+0000","last_change":"2023-09-12T14:55:56.961308+0000","last_active":"2023-09-12T14:55:59.189466+0000","last_peered":"2023-09-12T14:55:59.189466+0000","last_clean":"2023-09-12T14:55:59.189466+0000","last_became_active":"2023-09-12T14:55:56.960758+0000","last_became_peered":"2023-09-12T14:55:56.960758+0000","last_unstale":"2023-09-12T14:55:59.189466+0000","last_undegraded":"2023-09-12T14:55:59.189466+0000","last_fullsized":"2023-09-12T14:55:59.189466+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:32:13.749664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.528783+0000","last_change":"2023-09-12T14:55:56.544207+0000","last_active":"2023-09-12T14:55:57.528783+0000","last_peered":"2023-09-12T14:55:57.528783+0000","last_clean":"2023-09-12T14:55:57.528783+0000","last_became_active":"2023-09-12T14:55:56.543652+0000","last_became_peered":"2023-09-12T14:55:56.543652+0000","last_unstale":"2023-09-12T14:55:57.528783+0000","last_undegraded":"2023-09-12T14:55:57.528783+0000","last_fullsized":"2023-09-12T14:55:57.528783+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:53:40.126003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.911017+0000","last_change":"2023-09-12T14:55:56.935797+0000","last_active":"2023-09-12T14:55:57.911017+0000","last_peered":"2023-09-12T14:55:57.911017+0000","last_clean":"2023-09-12T14:55:57.911017+0000","last_became_active":"2023-09-12T14:55:56.934895+0000","last_became_peered":"2023-09-12T14:55:56.934895+0000","last_unstale":"2023-09-12T14:55:57.911017+0000","last_undegraded":"2023-09-12T14:55:57.911017+0000","last_fullsized":"2023-09-12T14:55:57.911017+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:26:20.168808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.528262+0000","last_change":"2023-09-12T14:55:56.541914+0000","last_active":"2023-09-12T14:55:57.528262+0000","last_peered":"2023-09-12T14:55:57.528262+0000","last_clean":"2023-09-12T14:55:57.528262+0000","last_became_active":"2023-09-12T14:55:56.541110+0000","last_became_peered":"2023-09-12T14:55:56.541110+0000","last_unstale":"2023-09-12T14:55:57.528262+0000","last_undegraded":"2023-09-12T14:55:57.528262+0000","last_fullsized":"2023-09-12T14:55:57.528262+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:27:22.656167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.909839+0000","last_change":"2023-09-12T14:55:56.944913+0000","last_active":"2023-09-12T14:55:57.909839+0000","last_peered":"2023-09-12T14:55:57.909839+0000","last_clean":"2023-09-12T14:55:57.909839+0000","last_became_active":"2023-09-12T14:55:56.942134+0000","last_became_peered":"2023-09-12T14:55:56.942134+0000","last_unstale":"2023-09-12T14:55:57.909839+0000","last_undegraded":"2023-09-12T14:55:57.909839+0000","last_fullsized":"2023-09-12T14:55:57.909839+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:50:32.960307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.903470+0000","last_change":"2023-09-12T14:55:56.531053+0000","last_active":"2023-09-12T14:55:56.903470+0000","last_peered":"2023-09-12T14:55:56.903470+0000","last_clean":"2023-09-12T14:55:56.903470+0000","last_became_active":"2023-09-12T14:55:56.530748+0000","last_became_peered":"2023-09-12T14:55:56.530748+0000","last_unstale":"2023-09-12T14:55:56.903470+0000","last_undegraded":"2023-09-12T14:55:56.903470+0000","last_fullsized":"2023-09-12T14:55:56.903470+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:50:31.955735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189427+0000","last_change":"2023-09-12T14:55:56.940009+0000","last_active":"2023-09-12T14:55:59.189427+0000","last_peered":"2023-09-12T14:55:59.189427+0000","last_clean":"2023-09-12T14:55:59.189427+0000","last_became_active":"2023-09-12T14:55:56.939402+0000","last_became_peered":"2023-09-12T14:55:56.939402+0000","last_unstale":"2023-09-12T14:55:59.189427+0000","last_undegraded":"2023-09-12T14:55:59.189427+0000","last_fullsized":"2023-09-12T14:55:59.189427+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:10:13.906110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.913244+0000","last_change":"2023-09-12T14:55:56.541874+0000","last_active":"2023-09-12T14:55:56.913244+0000","last_peered":"2023-09-12T14:55:56.913244+0000","last_clean":"2023-09-12T14:55:56.913244+0000","last_became_active":"2023-09-12T14:55:56.539725+0000","last_became_peered":"2023-09-12T14:55:56.539725+0000","last_unstale":"2023-09-12T14:55:56.913244+0000","last_undegraded":"2023-09-12T14:55:56.913244+0000","last_fullsized":"2023-09-12T14:55:56.913244+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:54:13.129205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.911751+0000","last_change":"2023-09-12T14:55:56.935986+0000","last_active":"2023-09-12T14:55:57.911751+0000","last_peered":"2023-09-12T14:55:57.911751+0000","last_clean":"2023-09-12T14:55:57.911751+0000","last_became_active":"2023-09-12T14:55:56.935145+0000","last_became_peered":"2023-09-12T14:55:56.935145+0000","last_unstale":"2023-09-12T14:55:57.911751+0000","last_undegraded":"2023-09-12T14:55:57.911751+0000","last_fullsized":"2023-09-12T14:55:57.911751+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:15:17.791216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.906351+0000","last_change":"2023-09-12T14:55:56.549758+0000","last_active":"2023-09-12T14:55:56.906351+0000","last_peered":"2023-09-12T14:55:56.906351+0000","last_clean":"2023-09-12T14:55:56.906351+0000","last_became_active":"2023-09-12T14:55:56.549407+0000","last_became_peered":"2023-09-12T14:55:56.549407+0000","last_unstale":"2023-09-12T14:55:56.906351+0000","last_undegraded":"2023-09-12T14:55:56.906351+0000","last_fullsized":"2023-09-12T14:55:56.906351+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:51:56.327209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.193650+0000","last_change":"2023-09-12T14:55:56.943011+0000","last_active":"2023-09-12T14:55:59.193650+0000","last_peered":"2023-09-12T14:55:59.193650+0000","last_clean":"2023-09-12T14:55:59.193650+0000","last_became_active":"2023-09-12T14:55:56.942313+0000","last_became_peered":"2023-09-12T14:55:56.942313+0000","last_unstale":"2023-09-12T14:55:59.193650+0000","last_undegraded":"2023-09-12T14:55:59.193650+0000","last_fullsized":"2023-09-12T14:55:59.193650+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:00:06.884652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.918809+0000","last_change":"2023-09-12T14:55:56.532566+0000","last_active":"2023-09-12T14:55:56.918809+0000","last_peered":"2023-09-12T14:55:56.918809+0000","last_clean":"2023-09-12T14:55:56.918809+0000","last_became_active":"2023-09-12T14:55:56.532122+0000","last_became_peered":"2023-09-12T14:55:56.532122+0000","last_unstale":"2023-09-12T14:55:56.918809+0000","last_undegraded":"2023-09-12T14:55:56.918809+0000","last_fullsized":"2023-09-12T14:55:56.918809+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:46:32.695633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.191443+0000","last_change":"2023-09-12T14:55:56.946251+0000","last_active":"2023-09-12T14:55:59.191443+0000","last_peered":"2023-09-12T14:55:59.191443+0000","last_clean":"2023-09-12T14:55:59.191443+0000","last_became_active":"2023-09-12T14:55:56.945833+0000","last_became_peered":"2023-09-12T14:55:56.945833+0000","last_unstale":"2023-09-12T14:55:59.191443+0000","last_undegraded":"2023-09-12T14:55:59.191443+0000","last_fullsized":"2023-09-12T14:55:59.191443+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:15:17.791216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.530322+0000","last_change":"2023-09-12T14:55:56.550717+0000","last_active":"2023-09-12T14:55:57.530322+0000","last_peered":"2023-09-12T14:55:57.530322+0000","last_clean":"2023-09-12T14:55:57.530322+0000","last_became_active":"2023-09-12T14:55:56.550055+0000","last_became_peered":"2023-09-12T14:55:56.550055+0000","last_unstale":"2023-09-12T14:55:57.530322+0000","last_undegraded":"2023-09-12T14:55:57.530322+0000","last_fullsized":"2023-09-12T14:55:57.530322+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:51:56.327209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.911356+0000","last_change":"2023-09-12T14:55:56.944565+0000","last_active":"2023-09-12T14:55:57.911356+0000","last_peered":"2023-09-12T14:55:57.911356+0000","last_clean":"2023-09-12T14:55:57.911356+0000","last_became_active":"2023-09-12T14:55:56.943807+0000","last_became_peered":"2023-09-12T14:55:56.943807+0000","last_unstale":"2023-09-12T14:55:57.911356+0000","last_undegraded":"2023-09-12T14:55:57.911356+0000","last_fullsized":"2023-09-12T14:55:57.911356+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:24:20.331640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.532314+0000","last_change":"2023-09-12T14:55:56.534785+0000","last_active":"2023-09-12T14:55:57.532314+0000","last_peered":"2023-09-12T14:55:57.532314+0000","last_clean":"2023-09-12T14:55:57.532314+0000","last_became_active":"2023-09-12T14:55:56.534401+0000","last_became_peered":"2023-09-12T14:55:56.534401+0000","last_unstale":"2023-09-12T14:55:57.532314+0000","last_undegraded":"2023-09-12T14:55:57.532314+0000","last_fullsized":"2023-09-12T14:55:57.532314+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:08:55.579592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.190836+0000","last_change":"2023-09-12T14:55:56.936770+0000","last_active":"2023-09-12T14:55:59.190836+0000","last_peered":"2023-09-12T14:55:59.190836+0000","last_clean":"2023-09-12T14:55:59.190836+0000","last_became_active":"2023-09-12T14:55:56.936011+0000","last_became_peered":"2023-09-12T14:55:56.936011+0000","last_unstale":"2023-09-12T14:55:59.190836+0000","last_undegraded":"2023-09-12T14:55:59.190836+0000","last_fullsized":"2023-09-12T14:55:59.190836+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:51:57.331781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.529740+0000","last_change":"2023-09-12T14:55:56.542724+0000","last_active":"2023-09-12T14:55:57.529740+0000","last_peered":"2023-09-12T14:55:57.529740+0000","last_clean":"2023-09-12T14:55:57.529740+0000","last_became_active":"2023-09-12T14:55:56.541958+0000","last_became_peered":"2023-09-12T14:55:56.541958+0000","last_unstale":"2023-09-12T14:55:57.529740+0000","last_undegraded":"2023-09-12T14:55:57.529740+0000","last_fullsized":"2023-09-12T14:55:57.529740+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:48:34.208772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.191555+0000","last_change":"2023-09-12T14:55:56.927415+0000","last_active":"2023-09-12T14:55:59.191555+0000","last_peered":"2023-09-12T14:55:59.191555+0000","last_clean":"2023-09-12T14:55:59.191555+0000","last_became_active":"2023-09-12T14:55:56.926767+0000","last_became_peered":"2023-09-12T14:55:56.926767+0000","last_unstale":"2023-09-12T14:55:59.191555+0000","last_undegraded":"2023-09-12T14:55:59.191555+0000","last_fullsized":"2023-09-12T14:55:59.191555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:51:11.665671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.915603+0000","last_change":"2023-09-12T14:55:56.533893+0000","last_active":"2023-09-12T14:55:56.915603+0000","last_peered":"2023-09-12T14:55:56.915603+0000","last_clean":"2023-09-12T14:55:56.915603+0000","last_became_active":"2023-09-12T14:55:56.533521+0000","last_became_peered":"2023-09-12T14:55:56.533521+0000","last_unstale":"2023-09-12T14:55:56.915603+0000","last_undegraded":"2023-09-12T14:55:56.915603+0000","last_fullsized":"2023-09-12T14:55:56.915603+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:50:31.955735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189448+0000","last_change":"2023-09-12T14:55:56.961621+0000","last_active":"2023-09-12T14:55:59.189448+0000","last_peered":"2023-09-12T14:55:59.189448+0000","last_clean":"2023-09-12T14:55:59.189448+0000","last_became_active":"2023-09-12T14:55:56.960906+0000","last_became_peered":"2023-09-12T14:55:56.960906+0000","last_unstale":"2023-09-12T14:55:59.189448+0000","last_undegraded":"2023-09-12T14:55:59.189448+0000","last_fullsized":"2023-09-12T14:55:59.189448+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:08:56.584164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.903112+0000","last_change":"2023-09-12T14:55:56.549738+0000","last_active":"2023-09-12T14:55:56.903112+0000","last_peered":"2023-09-12T14:55:56.903112+0000","last_clean":"2023-09-12T14:55:56.903112+0000","last_became_active":"2023-09-12T14:55:56.549335+0000","last_became_peered":"2023-09-12T14:55:56.549335+0000","last_unstale":"2023-09-12T14:55:56.903112+0000","last_undegraded":"2023-09-12T14:55:56.903112+0000","last_fullsized":"2023-09-12T14:55:56.903112+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:44:26.291246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:58.233939+0000","last_change":"2023-09-12T14:55:56.951267+0000","last_active":"2023-09-12T14:55:58.233939+0000","last_peered":"2023-09-12T14:55:58.233939+0000","last_clean":"2023-09-12T14:55:58.233939+0000","last_became_active":"2023-09-12T14:55:56.949172+0000","last_became_peered":"2023-09-12T14:55:56.949172+0000","last_unstale":"2023-09-12T14:55:58.233939+0000","last_undegraded":"2023-09-12T14:55:58.233939+0000","last_fullsized":"2023-09-12T14:55:58.233939+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:48:44.893608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.530290+0000","last_change":"2023-09-12T14:55:56.534769+0000","last_active":"2023-09-12T14:55:57.530290+0000","last_peered":"2023-09-12T14:55:57.530290+0000","last_clean":"2023-09-12T14:55:57.530290+0000","last_became_active":"2023-09-12T14:55:56.534371+0000","last_became_peered":"2023-09-12T14:55:56.534371+0000","last_unstale":"2023-09-12T14:55:57.530290+0000","last_undegraded":"2023-09-12T14:55:57.530290+0000","last_fullsized":"2023-09-12T14:55:57.530290+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:24:04.838737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.910240+0000","last_change":"2023-09-12T14:55:56.948577+0000","last_active":"2023-09-12T14:55:57.910240+0000","last_peered":"2023-09-12T14:55:57.910240+0000","last_clean":"2023-09-12T14:55:57.910240+0000","last_became_active":"2023-09-12T14:55:56.946473+0000","last_became_peered":"2023-09-12T14:55:56.946473+0000","last_unstale":"2023-09-12T14:55:57.910240+0000","last_undegraded":"2023-09-12T14:55:57.910240+0000","last_fullsized":"2023-09-12T14:55:57.910240+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:50:12.068597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.525763+0000","last_change":"2023-09-12T14:55:56.532423+0000","last_active":"2023-09-12T14:55:57.525763+0000","last_peered":"2023-09-12T14:55:57.525763+0000","last_clean":"2023-09-12T14:55:57.525763+0000","last_became_active":"2023-09-12T14:55:56.531971+0000","last_became_peered":"2023-09-12T14:55:56.531971+0000","last_unstale":"2023-09-12T14:55:57.525763+0000","last_undegraded":"2023-09-12T14:55:57.525763+0000","last_fullsized":"2023-09-12T14:55:57.525763+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:24:52.205479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.911747+0000","last_change":"2023-09-12T14:55:56.948863+0000","last_active":"2023-09-12T14:55:57.911747+0000","last_peered":"2023-09-12T14:55:57.911747+0000","last_clean":"2023-09-12T14:55:57.911747+0000","last_became_active":"2023-09-12T14:55:56.947965+0000","last_became_peered":"2023-09-12T14:55:56.947965+0000","last_unstale":"2023-09-12T14:55:57.911747+0000","last_undegraded":"2023-09-12T14:55:57.911747+0000","last_fullsized":"2023-09-12T14:55:57.911747+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:54:34.731553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.528557+0000","last_change":"2023-09-12T14:55:56.538492+0000","last_active":"2023-09-12T14:55:57.528557+0000","last_peered":"2023-09-12T14:55:57.528557+0000","last_clean":"2023-09-12T14:55:57.528557+0000","last_became_active":"2023-09-12T14:55:56.537724+0000","last_became_peered":"2023-09-12T14:55:56.537724+0000","last_unstale":"2023-09-12T14:55:57.528557+0000","last_undegraded":"2023-09-12T14:55:57.528557+0000","last_fullsized":"2023-09-12T14:55:57.528557+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:47:23.257700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.192004+0000","last_change":"2023-09-12T14:55:56.936455+0000","last_active":"2023-09-12T14:55:59.192004+0000","last_peered":"2023-09-12T14:55:59.192004+0000","last_clean":"2023-09-12T14:55:59.192004+0000","last_became_active":"2023-09-12T14:55:56.935777+0000","last_became_peered":"2023-09-12T14:55:56.935777+0000","last_unstale":"2023-09-12T14:55:59.192004+0000","last_undegraded":"2023-09-12T14:55:59.192004+0000","last_fullsized":"2023-09-12T14:55:59.192004+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:37:00.305524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.528588+0000","last_change":"2023-09-12T14:55:56.539779+0000","last_active":"2023-09-12T14:55:57.528588+0000","last_peered":"2023-09-12T14:55:57.528588+0000","last_clean":"2023-09-12T14:55:57.528588+0000","last_became_active":"2023-09-12T14:55:56.539014+0000","last_became_peered":"2023-09-12T14:55:56.539014+0000","last_unstale":"2023-09-12T14:55:57.528588+0000","last_undegraded":"2023-09-12T14:55:57.528588+0000","last_fullsized":"2023-09-12T14:55:57.528588+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:31:39.967705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.190689+0000","last_change":"2023-09-12T14:55:56.942708+0000","last_active":"2023-09-12T14:55:59.190689+0000","last_peered":"2023-09-12T14:55:59.190689+0000","last_clean":"2023-09-12T14:55:59.190689+0000","last_became_active":"2023-09-12T14:55:56.941801+0000","last_became_peered":"2023-09-12T14:55:56.941801+0000","last_unstale":"2023-09-12T14:55:59.190689+0000","last_undegraded":"2023-09-12T14:55:59.190689+0000","last_fullsized":"2023-09-12T14:55:59.190689+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:24:05.311518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.919648+0000","last_change":"2023-09-12T14:55:56.542930+0000","last_active":"2023-09-12T14:55:56.919648+0000","last_peered":"2023-09-12T14:55:56.919648+0000","last_clean":"2023-09-12T14:55:56.919648+0000","last_became_active":"2023-09-12T14:55:56.541322+0000","last_became_peered":"2023-09-12T14:55:56.541322+0000","last_unstale":"2023-09-12T14:55:56.919648+0000","last_undegraded":"2023-09-12T14:55:56.919648+0000","last_fullsized":"2023-09-12T14:55:56.919648+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:46:32.695633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189301+0000","last_change":"2023-09-12T14:55:56.945804+0000","last_active":"2023-09-12T14:55:59.189301+0000","last_peered":"2023-09-12T14:55:59.189301+0000","last_clean":"2023-09-12T14:55:59.189301+0000","last_became_active":"2023-09-12T14:55:56.945486+0000","last_became_peered":"2023-09-12T14:55:56.945486+0000","last_unstale":"2023-09-12T14:55:59.189301+0000","last_undegraded":"2023-09-12T14:55:59.189301+0000","last_fullsized":"2023-09-12T14:55:59.189301+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:53:41.130575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.528421+0000","last_change":"2023-09-12T14:55:56.520900+0000","last_active":"2023-09-12T14:55:57.528421+0000","last_peered":"2023-09-12T14:55:57.528421+0000","last_clean":"2023-09-12T14:55:57.528421+0000","last_became_active":"2023-09-12T14:55:56.515607+0000","last_became_peered":"2023-09-12T14:55:56.515607+0000","last_unstale":"2023-09-12T14:55:57.528421+0000","last_undegraded":"2023-09-12T14:55:57.528421+0000","last_fullsized":"2023-09-12T14:55:57.528421+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:44:26.291246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.190265+0000","last_change":"2023-09-12T14:55:56.924355+0000","last_active":"2023-09-12T14:55:59.190265+0000","last_peered":"2023-09-12T14:55:59.190265+0000","last_clean":"2023-09-12T14:55:59.190265+0000","last_became_active":"2023-09-12T14:55:56.924044+0000","last_became_peered":"2023-09-12T14:55:56.924044+0000","last_unstale":"2023-09-12T14:55:59.190265+0000","last_undegraded":"2023-09-12T14:55:59.190265+0000","last_fullsized":"2023-09-12T14:55:59.190265+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:30:21.532796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.929588+0000","last_change":"2023-09-12T14:55:56.538763+0000","last_active":"2023-09-12T14:55:56.929588+0000","last_peered":"2023-09-12T14:55:56.929588+0000","last_clean":"2023-09-12T14:55:56.929588+0000","last_became_active":"2023-09-12T14:55:56.538095+0000","last_became_peered":"2023-09-12T14:55:56.538095+0000","last_unstale":"2023-09-12T14:55:56.929588+0000","last_undegraded":"2023-09-12T14:55:56.929588+0000","last_fullsized":"2023-09-12T14:55:56.929588+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:27:22.656167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189875+0000","last_change":"2023-09-12T14:55:56.925305+0000","last_active":"2023-09-12T14:55:59.189875+0000","last_peered":"2023-09-12T14:55:59.189875+0000","last_clean":"2023-09-12T14:55:59.189875+0000","last_became_active":"2023-09-12T14:55:56.924770+0000","last_became_peered":"2023-09-12T14:55:56.924770+0000","last_unstale":"2023-09-12T14:55:59.189875+0000","last_undegraded":"2023-09-12T14:55:59.189875+0000","last_fullsized":"2023-09-12T14:55:59.189875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:48:35.213344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.913726+0000","last_change":"2023-09-12T14:55:56.543061+0000","last_active":"2023-09-12T14:55:56.913726+0000","last_peered":"2023-09-12T14:55:56.913726+0000","last_clean":"2023-09-12T14:55:56.913726+0000","last_became_active":"2023-09-12T14:55:56.542293+0000","last_became_peered":"2023-09-12T14:55:56.542293+0000","last_unstale":"2023-09-12T14:55:56.913726+0000","last_undegraded":"2023-09-12T14:55:56.913726+0000","last_fullsized":"2023-09-12T14:55:56.913726+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:24:36.430305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189688+0000","last_change":"2023-09-12T14:55:56.958745+0000","last_active":"2023-09-12T14:55:59.189688+0000","last_peered":"2023-09-12T14:55:59.189688+0000","last_clean":"2023-09-12T14:55:59.189688+0000","last_became_active":"2023-09-12T14:55:56.957778+0000","last_became_peered":"2023-09-12T14:55:56.957778+0000","last_unstale":"2023-09-12T14:55:59.189688+0000","last_undegraded":"2023-09-12T14:55:59.189688+0000","last_fullsized":"2023-09-12T14:55:59.189688+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:24:20.331640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.909971+0000","last_change":"2023-09-12T14:55:56.550760+0000","last_active":"2023-09-12T14:55:56.909971+0000","last_peered":"2023-09-12T14:55:56.909971+0000","last_clean":"2023-09-12T14:55:56.909971+0000","last_became_active":"2023-09-12T14:55:56.550171+0000","last_became_peered":"2023-09-12T14:55:56.550171+0000","last_unstale":"2023-09-12T14:55:56.909971+0000","last_undegraded":"2023-09-12T14:55:56.909971+0000","last_fullsized":"2023-09-12T14:55:56.909971+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:50:11.064026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.190810+0000","last_change":"2023-09-12T14:55:56.938612+0000","last_active":"2023-09-12T14:55:59.190810+0000","last_peered":"2023-09-12T14:55:59.190810+0000","last_clean":"2023-09-12T14:55:59.190810+0000","last_became_active":"2023-09-12T14:55:56.936787+0000","last_became_peered":"2023-09-12T14:55:56.936787+0000","last_unstale":"2023-09-12T14:55:59.190810+0000","last_undegraded":"2023-09-12T14:55:59.190810+0000","last_fullsized":"2023-09-12T14:55:59.190810+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:47:24.262272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.524768+0000","last_change":"2023-09-12T14:55:56.526327+0000","last_active":"2023-09-12T14:55:57.524768+0000","last_peered":"2023-09-12T14:55:57.524768+0000","last_clean":"2023-09-12T14:55:57.524768+0000","last_became_active":"2023-09-12T14:55:56.525989+0000","last_became_peered":"2023-09-12T14:55:56.525989+0000","last_unstale":"2023-09-12T14:55:57.524768+0000","last_undegraded":"2023-09-12T14:55:57.524768+0000","last_fullsized":"2023-09-12T14:55:57.524768+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:05:46.802102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.911917+0000","last_change":"2023-09-12T14:55:56.943720+0000","last_active":"2023-09-12T14:55:57.911917+0000","last_peered":"2023-09-12T14:55:57.911917+0000","last_clean":"2023-09-12T14:55:57.911917+0000","last_became_active":"2023-09-12T14:55:56.942886+0000","last_became_peered":"2023-09-12T14:55:56.942886+0000","last_unstale":"2023-09-12T14:55:57.911917+0000","last_undegraded":"2023-09-12T14:55:57.911917+0000","last_fullsized":"2023-09-12T14:55:57.911917+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:51:11.665671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.903471+0000","last_change":"2023-09-12T14:55:56.528396+0000","last_active":"2023-09-12T14:55:57.903471+0000","last_peered":"2023-09-12T14:55:57.903471+0000","last_clean":"2023-09-12T14:55:57.903471+0000","last_became_active":"2023-09-12T14:55:56.527993+0000","last_became_peered":"2023-09-12T14:55:56.527993+0000","last_unstale":"2023-09-12T14:55:57.903471+0000","last_undegraded":"2023-09-12T14:55:57.903471+0000","last_fullsized":"2023-09-12T14:55:57.903471+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:27:22.656167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.903268+0000","last_change":"2023-09-12T14:55:56.938713+0000","last_active":"2023-09-12T14:55:57.903268+0000","last_peered":"2023-09-12T14:55:57.903268+0000","last_clean":"2023-09-12T14:55:57.903268+0000","last_became_active":"2023-09-12T14:55:56.937836+0000","last_became_peered":"2023-09-12T14:55:56.937836+0000","last_unstale":"2023-09-12T14:55:57.903268+0000","last_undegraded":"2023-09-12T14:55:57.903268+0000","last_fullsized":"2023-09-12T14:55:57.903268+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:50:12.068597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.915820+0000","last_change":"2023-09-12T14:55:56.531043+0000","last_active":"2023-09-12T14:55:56.915820+0000","last_peered":"2023-09-12T14:55:56.915820+0000","last_clean":"2023-09-12T14:55:56.915820+0000","last_became_active":"2023-09-12T14:55:56.530557+0000","last_became_peered":"2023-09-12T14:55:56.530557+0000","last_unstale":"2023-09-12T14:55:56.915820+0000","last_undegraded":"2023-09-12T14:55:56.915820+0000","last_fullsized":"2023-09-12T14:55:56.915820+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:50:11.064026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:58.233954+0000","last_change":"2023-09-12T14:55:56.944558+0000","last_active":"2023-09-12T14:55:58.233954+0000","last_peered":"2023-09-12T14:55:58.233954+0000","last_clean":"2023-09-12T14:55:58.233954+0000","last_became_active":"2023-09-12T14:55:56.943774+0000","last_became_peered":"2023-09-12T14:55:56.943774+0000","last_unstale":"2023-09-12T14:55:58.233954+0000","last_undegraded":"2023-09-12T14:55:58.233954+0000","last_fullsized":"2023-09-12T14:55:58.233954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:24:53.210051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.922923+0000","last_change":"2023-09-12T14:55:56.536269+0000","last_active":"2023-09-12T14:55:56.922923+0000","last_peered":"2023-09-12T14:55:56.922923+0000","last_clean":"2023-09-12T14:55:56.922923+0000","last_became_active":"2023-09-12T14:55:56.535875+0000","last_became_peered":"2023-09-12T14:55:56.535875+0000","last_unstale":"2023-09-12T14:55:56.922923+0000","last_undegraded":"2023-09-12T14:55:56.922923+0000","last_fullsized":"2023-09-12T14:55:56.922923+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:51:37.543569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.903377+0000","last_change":"2023-09-12T14:55:56.945018+0000","last_active":"2023-09-12T14:55:57.903377+0000","last_peered":"2023-09-12T14:55:57.903377+0000","last_clean":"2023-09-12T14:55:57.903377+0000","last_became_active":"2023-09-12T14:55:56.944343+0000","last_became_peered":"2023-09-12T14:55:56.944343+0000","last_unstale":"2023-09-12T14:55:57.903377+0000","last_undegraded":"2023-09-12T14:55:57.903377+0000","last_fullsized":"2023-09-12T14:55:57.903377+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:24:53.210051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.915674+0000","last_change":"2023-09-12T14:55:56.536191+0000","last_active":"2023-09-12T14:55:56.915674+0000","last_peered":"2023-09-12T14:55:56.915674+0000","last_clean":"2023-09-12T14:55:56.915674+0000","last_became_active":"2023-09-12T14:55:56.535847+0000","last_became_peered":"2023-09-12T14:55:56.535847+0000","last_unstale":"2023-09-12T14:55:56.915674+0000","last_undegraded":"2023-09-12T14:55:56.915674+0000","last_fullsized":"2023-09-12T14:55:56.915674+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:24:04.838737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189930+0000","last_change":"2023-09-12T14:55:56.942522+0000","last_active":"2023-09-12T14:55:59.189930+0000","last_peered":"2023-09-12T14:55:59.189930+0000","last_clean":"2023-09-12T14:55:59.189930+0000","last_became_active":"2023-09-12T14:55:56.941639+0000","last_became_peered":"2023-09-12T14:55:56.941639+0000","last_unstale":"2023-09-12T14:55:59.189930+0000","last_undegraded":"2023-09-12T14:55:59.189930+0000","last_fullsized":"2023-09-12T14:55:59.189930+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:54:34.731553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.914061+0000","last_change":"2023-09-12T14:55:56.541270+0000","last_active":"2023-09-12T14:55:56.914061+0000","last_peered":"2023-09-12T14:55:56.914061+0000","last_clean":"2023-09-12T14:55:56.914061+0000","last_became_active":"2023-09-12T14:55:56.540773+0000","last_became_peered":"2023-09-12T14:55:56.540773+0000","last_unstale":"2023-09-12T14:55:56.914061+0000","last_undegraded":"2023-09-12T14:55:56.914061+0000","last_fullsized":"2023-09-12T14:55:56.914061+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:15:12.274569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.909161+0000","last_change":"2023-09-12T14:55:56.937532+0000","last_active":"2023-09-12T14:55:57.909161+0000","last_peered":"2023-09-12T14:55:57.909161+0000","last_clean":"2023-09-12T14:55:57.909161+0000","last_became_active":"2023-09-12T14:55:56.936799+0000","last_became_peered":"2023-09-12T14:55:56.936799+0000","last_unstale":"2023-09-12T14:55:57.909161+0000","last_undegraded":"2023-09-12T14:55:57.909161+0000","last_fullsized":"2023-09-12T14:55:57.909161+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:51:38.548141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.903377+0000","last_change":"2023-09-12T14:55:56.549806+0000","last_active":"2023-09-12T14:55:56.903377+0000","last_peered":"2023-09-12T14:55:56.903377+0000","last_clean":"2023-09-12T14:55:56.903377+0000","last_became_active":"2023-09-12T14:55:56.549413+0000","last_became_peered":"2023-09-12T14:55:56.549413+0000","last_unstale":"2023-09-12T14:55:56.903377+0000","last_undegraded":"2023-09-12T14:55:56.903377+0000","last_fullsized":"2023-09-12T14:55:56.903377+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:51:37.543569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189609+0000","last_change":"2023-09-12T14:55:56.954426+0000","last_active":"2023-09-12T14:55:59.189609+0000","last_peered":"2023-09-12T14:55:59.189609+0000","last_clean":"2023-09-12T14:55:59.189609+0000","last_became_active":"2023-09-12T14:55:56.953811+0000","last_became_peered":"2023-09-12T14:55:56.953811+0000","last_unstale":"2023-09-12T14:55:59.189609+0000","last_undegraded":"2023-09-12T14:55:59.189609+0000","last_fullsized":"2023-09-12T14:55:59.189609+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:26:20.168808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.929496+0000","last_change":"2023-09-12T14:55:56.545462+0000","last_active":"2023-09-12T14:55:56.929496+0000","last_peered":"2023-09-12T14:55:56.929496+0000","last_clean":"2023-09-12T14:55:56.929496+0000","last_became_active":"2023-09-12T14:55:56.544959+0000","last_became_peered":"2023-09-12T14:55:56.544959+0000","last_unstale":"2023-09-12T14:55:56.929496+0000","last_undegraded":"2023-09-12T14:55:56.929496+0000","last_fullsized":"2023-09-12T14:55:56.929496+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:46:32.695633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":176,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.923028+0000","last_change":"2023-09-12T14:54:46.965446+0000","last_active":"2023-09-12T14:55:56.923028+0000","last_peered":"2023-09-12T14:55:56.923028+0000","last_clean":"2023-09-12T14:55:56.923028+0000","last_became_active":"2023-09-12T14:54:46.963236+0000","last_became_peered":"2023-09-12T14:54:46.963236+0000","last_unstale":"2023-09-12T14:55:56.923028+0000","last_undegraded":"2023-09-12T14:55:56.923028+0000","last_fullsized":"2023-09-12T14:55:56.923028+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:45.804232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:45.804232+0000","last_clean_scrub_stamp":"2023-09-12T14:54:45.804232+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:24:45.670513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":737386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.95299999999999996}]},{"osd":1,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":1.119}]},{"osd":2,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.145},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.05}]},{"osd":3,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.82699999999999996}]},{"osd":4,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.071}]},{"osd":5,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.70199999999999996}]},{"osd":6,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.76600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.68500000000000005}]},{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.88600000000000001}]},{"osd":2,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.628},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":1.161}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.052},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.73499999999999999}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":1.2150000000000001}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":1.115},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":1.2969999999999999}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.01},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":1.347}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":1.0680000000000001}]},{"osd":1,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.014}]},{"osd":2,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.222},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.89900000000000002}]},{"osd":3,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":1.1759999999999999},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":1.337}]},{"osd":4,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.3640000000000001}]},{"osd":6,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.516,"5min":2.516,"15min":2.516},"last":1.103},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":1.292}]},{"osd":7,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.248},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":1.391}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":380928,"data_stored":147032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.88400000000000001}]},{"osd":1,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.83499999999999996}]},{"osd":2,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":0.96799999999999997}]},{"osd":3,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":1.0069999999999999}]},{"osd":5,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.1739999999999999},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.94399999999999995}]},{"osd":6,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":1.0880000000000001}]},{"osd":7,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":1.044},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":1.133}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.38},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.46700000000000003}]},{"osd":1,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.436}]},{"osd":2,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.54200000000000004}]},{"osd":4,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.81100000000000005}]},{"osd":5,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.622},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.73199999999999998}]},{"osd":6,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.871},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.66100000000000003}]},{"osd":7,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.67000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":146998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.66500000000000004}]},{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.621},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":1.0549999999999999}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.75900000000000001}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.80700000000000005}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.91200000000000003}]},{"osd":6,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.1989999999999998,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":1.0840000000000001}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":1.02}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.75800000000000001}]},{"osd":2,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.84899999999999998}]},{"osd":3,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.67600000000000005}]},{"osd":4,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":1.3440000000000001},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.179}]},{"osd":5,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":1.292}]},{"osd":6,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":1.208}]},{"osd":7,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":1.254},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.0860000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":962560,"data_stored":736277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.372},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.312}]},{"osd":2,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.38500000000000001}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.505},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.46400000000000002}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.52900000000000003}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":0.59899999999999998},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":0.78800000000000003}]},{"osd":6,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.73799999999999999}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.755},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.92900000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-12T14:56:02.242 INFO:tasks.ceph.mds.a.smithi131.stderr:2023-09-12T14:56:02.240+0000 7f656f5ec640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-12T14:56:04.999 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-12T14:56:05.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.136+0000 7fad46946640 1 -- 172.21.15.131:0/1048597310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad401574d0 msgr2=0x7fad400cacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:05.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.136+0000 7fad46946640 1 --2- 172.21.15.131:0/1048597310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad401574d0 0x7fad400cacd0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fad30009840 tx=0x7fad3001c7d0 comp rx=0 tx=0).stop 2023-09-12T14:56:05.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.136+0000 7fad46946640 1 -- 172.21.15.131:0/1048597310 shutdown_connections 2023-09-12T14:56:05.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.136+0000 7fad46946640 1 --2- 172.21.15.131:0/1048597310 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad400c9aa0 0x7fad400af9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:05.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.136+0000 7fad46946640 1 --2- 172.21.15.131:0/1048597310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad401574d0 0x7fad400cacd0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:05.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.136+0000 7fad46946640 1 -- 172.21.15.131:0/1048597310 >> 172.21.15.131:0/1048597310 conn(0x7fad40069840 msgr2=0x7fad400cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:05.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.136+0000 7fad46946640 1 -- 172.21.15.131:0/1048597310 shutdown_connections 2023-09-12T14:56:05.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.136+0000 7fad46946640 1 -- 172.21.15.131:0/1048597310 wait complete. 2023-09-12T14:56:05.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.136+0000 7fad46946640 1 Processor -- start 2023-09-12T14:56:05.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.136+0000 7fad46946640 1 -- start start 2023-09-12T14:56:05.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.136+0000 7fad46946640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad400b2600 0x7fad40159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:05.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.136+0000 7fad46946640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad400c9aa0 0x7fad4014c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:05.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.136+0000 7fad46946640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad401574d0 0x7fad4014dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:05.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.136+0000 7fad46946640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4014cbf0 con 0x7fad400c9aa0 2023-09-12T14:56:05.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.136+0000 7fad46946640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4014cde0 con 0x7fad400b2600 2023-09-12T14:56:05.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.136+0000 7fad46946640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad4014d0c0 con 0x7fad401574d0 2023-09-12T14:56:05.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.136+0000 7fad3f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad400c9aa0 0x7fad4014c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:05.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.136+0000 7fad3f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad400c9aa0 0x7fad4014c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37444/0 (socket says 172.21.15.131:37444) 2023-09-12T14:56:05.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.136+0000 7fad3f7fe640 1 -- 172.21.15.131:0/1176371538 learned_addr learned my addr 172.21.15.131:0/1176371538 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:05.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.136+0000 7fad44ee3640 1 --2- 172.21.15.131:0/1176371538 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad401574d0 0x7fad4014dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:05.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.136+0000 7fad3ffff640 1 --2- 172.21.15.131:0/1176371538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad400b2600 0x7fad40159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:05.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.136+0000 7fad3f7fe640 1 -- 172.21.15.131:0/1176371538 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad401574d0 msgr2=0x7fad4014dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:05.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.136+0000 7fad3f7fe640 1 --2- 172.21.15.131:0/1176371538 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad401574d0 0x7fad4014dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:05.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.136+0000 7fad3f7fe640 1 -- 172.21.15.131:0/1176371538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad400b2600 msgr2=0x7fad40159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:05.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.136+0000 7fad3f7fe640 1 --2- 172.21.15.131:0/1176371538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad400b2600 0x7fad40159a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:05.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.136+0000 7fad3f7fe640 1 -- 172.21.15.131:0/1176371538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad300094a0 con 0x7fad400c9aa0 2023-09-12T14:56:05.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.136+0000 7fad3f7fe640 1 --2- 172.21.15.131:0/1176371538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad400c9aa0 0x7fad4014c6b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fad3400c480 tx=0x7fad3400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:05.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.140+0000 7fad3cff9640 1 -- 172.21.15.131:0/1176371538 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad34012830 con 0x7fad400c9aa0 2023-09-12T14:56:05.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.140+0000 7fad3cff9640 1 -- 172.21.15.131:0/1176371538 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fad34012a80 con 0x7fad400c9aa0 2023-09-12T14:56:05.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.140+0000 7fad46946640 1 -- 172.21.15.131:0/1176371538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad4014d3a0 con 0x7fad400c9aa0 2023-09-12T14:56:05.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.140+0000 7fad3cff9640 1 -- 172.21.15.131:0/1176371538 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad34012e70 con 0x7fad400c9aa0 2023-09-12T14:56:05.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.140+0000 7fad46946640 1 -- 172.21.15.131:0/1176371538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad4014d960 con 0x7fad400c9aa0 2023-09-12T14:56:05.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.140+0000 7fad46946640 1 -- 172.21.15.131:0/1176371538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad40146f30 con 0x7fad400c9aa0 2023-09-12T14:56:05.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.140+0000 7fad3cff9640 1 -- 172.21.15.131:0/1176371538 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fad3401a3b0 con 0x7fad400c9aa0 2023-09-12T14:56:05.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.140+0000 7fad3cff9640 1 --2- 172.21.15.131:0/1176371538 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fad2c065b20 0x7fad2c067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:05.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.140+0000 7fad3ffff640 1 --2- 172.21.15.131:0/1176371538 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fad2c065b20 0x7fad2c067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:05.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.140+0000 7fad3cff9640 1 -- 172.21.15.131:0/1176371538 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad340833a0 con 0x7fad400c9aa0 2023-09-12T14:56:05.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.140+0000 7fad3ffff640 1 --2- 172.21.15.131:0/1176371538 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fad2c065b20 0x7fad2c067fe0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fad300af000 tx=0x7fad3001b010 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:05.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.144+0000 7fad3cff9640 1 -- 172.21.15.131:0/1176371538 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fad40146f30 con 0x7fad400c9aa0 2023-09-12T14:56:05.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.292+0000 7fad46946640 1 -- 172.21.15.131:0/1176371538 --> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fad40146f10 con 0x7fad2c065b20 2023-09-12T14:56:05.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.304+0000 7fad3cff9640 1 -- 172.21.15.131:0/1176371538 <== mgr.4200 v2:172.21.15.131:6832/2298103651 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287773 (secure 0 0 0) 0x7fad40146f10 con 0x7fad2c065b20 2023-09-12T14:56:05.310 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:56:05.311 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-09-12T14:56:05.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.308+0000 7fad46946640 1 -- 172.21.15.131:0/1176371538 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fad2c065b20 msgr2=0x7fad2c067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:05.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.308+0000 7fad46946640 1 --2- 172.21.15.131:0/1176371538 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fad2c065b20 0x7fad2c067fe0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fad300af000 tx=0x7fad3001b010 comp rx=0 tx=0).stop 2023-09-12T14:56:05.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.308+0000 7fad46946640 1 -- 172.21.15.131:0/1176371538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad400c9aa0 msgr2=0x7fad4014c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:05.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.308+0000 7fad46946640 1 --2- 172.21.15.131:0/1176371538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad400c9aa0 0x7fad4014c6b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fad3400c480 tx=0x7fad3400c940 comp rx=0 tx=0).stop 2023-09-12T14:56:05.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.308+0000 7fad46946640 1 -- 172.21.15.131:0/1176371538 shutdown_connections 2023-09-12T14:56:05.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.308+0000 7fad46946640 1 --2- 172.21.15.131:0/1176371538 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fad2c065b20 0x7fad2c067fe0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:05.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.308+0000 7fad46946640 1 --2- 172.21.15.131:0/1176371538 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad401574d0 0x7fad4014dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:05.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.308+0000 7fad46946640 1 --2- 172.21.15.131:0/1176371538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad400c9aa0 0x7fad4014c6b0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:05.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.308+0000 7fad46946640 1 --2- 172.21.15.131:0/1176371538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad400b2600 0x7fad40159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:05.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.308+0000 7fad46946640 1 -- 172.21.15.131:0/1176371538 >> 172.21.15.131:0/1176371538 conn(0x7fad40069840 msgr2=0x7fad400cc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:05.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.308+0000 7fad46946640 1 -- 172.21.15.131:0/1176371538 shutdown_connections 2023-09-12T14:56:05.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.308+0000 7fad46946640 1 -- 172.21.15.131:0/1176371538 wait complete. 2023-09-12T14:56:05.326 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2023-09-12T14:56:03.597207+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220312,"kb_used_data":4132,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754408,"statfs":{"total":773094113280,"available":772868513792,"internally_reserved":0,"allocated":4231168,"data_stored":2359961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":66,"apply_latency_ms":66,"commit_latency_ns":66000000,"apply_latency_ns":66000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.089736"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:58.234088+0000","last_change":"2023-09-12T14:55:56.947127+0000","last_active":"2023-09-12T14:55:58.234088+0000","last_peered":"2023-09-12T14:55:58.234088+0000","last_clean":"2023-09-12T14:55:58.234088+0000","last_became_active":"2023-09-12T14:55:56.946228+0000","last_became_peered":"2023-09-12T14:55:56.946228+0000","last_unstale":"2023-09-12T14:55:58.234088+0000","last_undegraded":"2023-09-12T14:55:58.234088+0000","last_fullsized":"2023-09-12T14:55:58.234088+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:05:47.806674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.193417+0000","last_change":"2023-09-12T14:55:56.927280+0000","last_active":"2023-09-12T14:55:59.193417+0000","last_peered":"2023-09-12T14:55:59.193417+0000","last_clean":"2023-09-12T14:55:59.193417+0000","last_became_active":"2023-09-12T14:55:56.924706+0000","last_became_peered":"2023-09-12T14:55:56.924706+0000","last_unstale":"2023-09-12T14:55:59.193417+0000","last_undegraded":"2023-09-12T14:55:59.193417+0000","last_fullsized":"2023-09-12T14:55:59.193417+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:31:58.066578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.911466+0000","last_change":"2023-09-12T14:55:56.940406+0000","last_active":"2023-09-12T14:55:57.911466+0000","last_peered":"2023-09-12T14:55:57.911466+0000","last_clean":"2023-09-12T14:55:57.911466+0000","last_became_active":"2023-09-12T14:55:56.939600+0000","last_became_peered":"2023-09-12T14:55:56.939600+0000","last_unstale":"2023-09-12T14:55:57.911466+0000","last_undegraded":"2023-09-12T14:55:57.911466+0000","last_fullsized":"2023-09-12T14:55:57.911466+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:17:33.154086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.191800+0000","last_change":"2023-09-12T14:55:56.950303+0000","last_active":"2023-09-12T14:55:59.191800+0000","last_peered":"2023-09-12T14:55:59.191800+0000","last_clean":"2023-09-12T14:55:59.191800+0000","last_became_active":"2023-09-12T14:55:56.949704+0000","last_became_peered":"2023-09-12T14:55:56.949704+0000","last_unstale":"2023-09-12T14:55:59.191800+0000","last_undegraded":"2023-09-12T14:55:59.191800+0000","last_fullsized":"2023-09-12T14:55:59.191800+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:08:56.584164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189875+0000","last_change":"2023-09-12T14:55:56.920821+0000","last_active":"2023-09-12T14:55:59.189875+0000","last_peered":"2023-09-12T14:55:59.189875+0000","last_clean":"2023-09-12T14:55:59.189875+0000","last_became_active":"2023-09-12T14:55:56.920251+0000","last_became_peered":"2023-09-12T14:55:56.920251+0000","last_unstale":"2023-09-12T14:55:59.189875+0000","last_undegraded":"2023-09-12T14:55:59.189875+0000","last_fullsized":"2023-09-12T14:55:59.189875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:12:36.768826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189089+0000","last_change":"2023-09-12T14:55:56.939064+0000","last_active":"2023-09-12T14:55:59.189089+0000","last_peered":"2023-09-12T14:55:59.189089+0000","last_clean":"2023-09-12T14:55:59.189089+0000","last_became_active":"2023-09-12T14:55:56.938751+0000","last_became_peered":"2023-09-12T14:55:56.938751+0000","last_unstale":"2023-09-12T14:55:59.189089+0000","last_undegraded":"2023-09-12T14:55:59.189089+0000","last_fullsized":"2023-09-12T14:55:59.189089+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:17:33.154086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189848+0000","last_change":"2023-09-12T14:55:56.958364+0000","last_active":"2023-09-12T14:55:59.189848+0000","last_peered":"2023-09-12T14:55:59.189848+0000","last_clean":"2023-09-12T14:55:59.189848+0000","last_became_active":"2023-09-12T14:55:56.957011+0000","last_became_peered":"2023-09-12T14:55:56.957011+0000","last_unstale":"2023-09-12T14:55:59.189848+0000","last_undegraded":"2023-09-12T14:55:59.189848+0000","last_fullsized":"2023-09-12T14:55:59.189848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:53:01.273053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.188843+0000","last_change":"2023-09-12T14:55:56.940471+0000","last_active":"2023-09-12T14:55:59.188843+0000","last_peered":"2023-09-12T14:55:59.188843+0000","last_clean":"2023-09-12T14:55:59.188843+0000","last_became_active":"2023-09-12T14:55:56.939686+0000","last_became_peered":"2023-09-12T14:55:56.939686+0000","last_unstale":"2023-09-12T14:55:59.188843+0000","last_undegraded":"2023-09-12T14:55:59.188843+0000","last_fullsized":"2023-09-12T14:55:59.188843+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:10:13.906110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.910516+0000","last_change":"2023-09-12T14:55:56.951450+0000","last_active":"2023-09-12T14:55:57.910516+0000","last_peered":"2023-09-12T14:55:57.910516+0000","last_clean":"2023-09-12T14:55:57.910516+0000","last_became_active":"2023-09-12T14:55:56.950607+0000","last_became_peered":"2023-09-12T14:55:56.950607+0000","last_unstale":"2023-09-12T14:55:57.910516+0000","last_undegraded":"2023-09-12T14:55:57.910516+0000","last_fullsized":"2023-09-12T14:55:57.910516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:05:47.806674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.191986+0000","last_change":"2023-09-12T14:55:56.938513+0000","last_active":"2023-09-12T14:55:59.191986+0000","last_peered":"2023-09-12T14:55:59.191986+0000","last_clean":"2023-09-12T14:55:59.191986+0000","last_became_active":"2023-09-12T14:55:56.935930+0000","last_became_peered":"2023-09-12T14:55:56.935930+0000","last_unstale":"2023-09-12T14:55:59.191986+0000","last_undegraded":"2023-09-12T14:55:59.191986+0000","last_fullsized":"2023-09-12T14:55:59.191986+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:32:13.749664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189504+0000","last_change":"2023-09-12T14:55:56.955591+0000","last_active":"2023-09-12T14:55:59.189504+0000","last_peered":"2023-09-12T14:55:59.189504+0000","last_clean":"2023-09-12T14:55:59.189504+0000","last_became_active":"2023-09-12T14:55:56.954971+0000","last_became_peered":"2023-09-12T14:55:56.954971+0000","last_unstale":"2023-09-12T14:55:59.189504+0000","last_undegraded":"2023-09-12T14:55:59.189504+0000","last_fullsized":"2023-09-12T14:55:59.189504+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:37:00.305524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.911366+0000","last_change":"2023-09-12T14:55:56.937413+0000","last_active":"2023-09-12T14:55:57.911366+0000","last_peered":"2023-09-12T14:55:57.911366+0000","last_clean":"2023-09-12T14:55:57.911366+0000","last_became_active":"2023-09-12T14:55:56.936637+0000","last_became_peered":"2023-09-12T14:55:56.936637+0000","last_unstale":"2023-09-12T14:55:57.911366+0000","last_undegraded":"2023-09-12T14:55:57.911366+0000","last_fullsized":"2023-09-12T14:55:57.911366+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:35:17.945837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.190266+0000","last_change":"2023-09-12T14:55:56.939993+0000","last_active":"2023-09-12T14:55:59.190266+0000","last_peered":"2023-09-12T14:55:59.190266+0000","last_clean":"2023-09-12T14:55:59.190266+0000","last_became_active":"2023-09-12T14:55:56.939681+0000","last_became_peered":"2023-09-12T14:55:56.939681+0000","last_unstale":"2023-09-12T14:55:59.190266+0000","last_undegraded":"2023-09-12T14:55:59.190266+0000","last_fullsized":"2023-09-12T14:55:59.190266+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:31:40.972277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189873+0000","last_change":"2023-09-12T14:55:56.919959+0000","last_active":"2023-09-12T14:55:59.189873+0000","last_peered":"2023-09-12T14:55:59.189873+0000","last_clean":"2023-09-12T14:55:59.189873+0000","last_became_active":"2023-09-12T14:55:56.919251+0000","last_became_peered":"2023-09-12T14:55:56.919251+0000","last_unstale":"2023-09-12T14:55:59.189873+0000","last_undegraded":"2023-09-12T14:55:59.189873+0000","last_fullsized":"2023-09-12T14:55:59.189873+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:53:01.273053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.909815+0000","last_change":"2023-09-12T14:55:56.946665+0000","last_active":"2023-09-12T14:55:57.909815+0000","last_peered":"2023-09-12T14:55:57.909815+0000","last_clean":"2023-09-12T14:55:57.909815+0000","last_became_active":"2023-09-12T14:55:56.945745+0000","last_became_peered":"2023-09-12T14:55:56.945745+0000","last_unstale":"2023-09-12T14:55:57.909815+0000","last_undegraded":"2023-09-12T14:55:57.909815+0000","last_fullsized":"2023-09-12T14:55:57.909815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:37:00.305524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189744+0000","last_change":"2023-09-12T14:55:56.944586+0000","last_active":"2023-09-12T14:55:59.189744+0000","last_peered":"2023-09-12T14:55:59.189744+0000","last_clean":"2023-09-12T14:55:59.189744+0000","last_became_active":"2023-09-12T14:55:56.943452+0000","last_became_peered":"2023-09-12T14:55:56.943452+0000","last_unstale":"2023-09-12T14:55:59.189744+0000","last_undegraded":"2023-09-12T14:55:59.189744+0000","last_fullsized":"2023-09-12T14:55:59.189744+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:17:33.154086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189888+0000","last_change":"2023-09-12T14:55:56.957246+0000","last_active":"2023-09-12T14:55:59.189888+0000","last_peered":"2023-09-12T14:55:59.189888+0000","last_clean":"2023-09-12T14:55:59.189888+0000","last_became_active":"2023-09-12T14:55:56.956737+0000","last_became_peered":"2023-09-12T14:55:56.956737+0000","last_unstale":"2023-09-12T14:55:59.189888+0000","last_undegraded":"2023-09-12T14:55:59.189888+0000","last_fullsized":"2023-09-12T14:55:59.189888+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:54:34.731553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.193648+0000","last_change":"2023-09-12T14:55:56.944981+0000","last_active":"2023-09-12T14:55:59.193648+0000","last_peered":"2023-09-12T14:55:59.193648+0000","last_clean":"2023-09-12T14:55:59.193648+0000","last_became_active":"2023-09-12T14:55:56.944227+0000","last_became_peered":"2023-09-12T14:55:56.944227+0000","last_unstale":"2023-09-12T14:55:59.193648+0000","last_undegraded":"2023-09-12T14:55:59.193648+0000","last_fullsized":"2023-09-12T14:55:59.193648+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:21:38.490784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.191590+0000","last_change":"2023-09-12T14:55:56.942243+0000","last_active":"2023-09-12T14:55:59.191590+0000","last_peered":"2023-09-12T14:55:59.191590+0000","last_clean":"2023-09-12T14:55:59.191590+0000","last_became_active":"2023-09-12T14:55:56.941715+0000","last_became_peered":"2023-09-12T14:55:56.941715+0000","last_unstale":"2023-09-12T14:55:59.191590+0000","last_undegraded":"2023-09-12T14:55:59.191590+0000","last_fullsized":"2023-09-12T14:55:59.191590+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:47:13.451453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189830+0000","last_change":"2023-09-12T14:55:56.946050+0000","last_active":"2023-09-12T14:55:59.189830+0000","last_peered":"2023-09-12T14:55:59.189830+0000","last_clean":"2023-09-12T14:55:59.189830+0000","last_became_active":"2023-09-12T14:55:56.945621+0000","last_became_peered":"2023-09-12T14:55:56.945621+0000","last_unstale":"2023-09-12T14:55:59.189830+0000","last_undegraded":"2023-09-12T14:55:59.189830+0000","last_fullsized":"2023-09-12T14:55:59.189830+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:54:06.886063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.910461+0000","last_change":"2023-09-12T14:55:56.932342+0000","last_active":"2023-09-12T14:55:57.910461+0000","last_peered":"2023-09-12T14:55:57.910461+0000","last_clean":"2023-09-12T14:55:57.910461+0000","last_became_active":"2023-09-12T14:55:56.931596+0000","last_became_peered":"2023-09-12T14:55:56.931596+0000","last_unstale":"2023-09-12T14:55:57.910461+0000","last_undegraded":"2023-09-12T14:55:57.910461+0000","last_fullsized":"2023-09-12T14:55:57.910461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:24:53.210051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.909956+0000","last_change":"2023-09-12T14:55:56.937472+0000","last_active":"2023-09-12T14:55:57.909956+0000","last_peered":"2023-09-12T14:55:57.909956+0000","last_clean":"2023-09-12T14:55:57.909956+0000","last_became_active":"2023-09-12T14:55:56.936724+0000","last_became_peered":"2023-09-12T14:55:56.936724+0000","last_unstale":"2023-09-12T14:55:57.909956+0000","last_undegraded":"2023-09-12T14:55:57.909956+0000","last_fullsized":"2023-09-12T14:55:57.909956+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:24:05.843309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.903393+0000","last_change":"2023-09-12T14:55:56.941598+0000","last_active":"2023-09-12T14:55:57.903393+0000","last_peered":"2023-09-12T14:55:57.903393+0000","last_clean":"2023-09-12T14:55:57.903393+0000","last_became_active":"2023-09-12T14:55:56.941328+0000","last_became_peered":"2023-09-12T14:55:56.941328+0000","last_unstale":"2023-09-12T14:55:57.903393+0000","last_undegraded":"2023-09-12T14:55:57.903393+0000","last_fullsized":"2023-09-12T14:55:57.903393+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:05:47.806674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.909123+0000","last_change":"2023-09-12T14:55:56.920070+0000","last_active":"2023-09-12T14:55:57.909123+0000","last_peered":"2023-09-12T14:55:57.909123+0000","last_clean":"2023-09-12T14:55:57.909123+0000","last_became_active":"2023-09-12T14:55:56.919314+0000","last_became_peered":"2023-09-12T14:55:56.919314+0000","last_unstale":"2023-09-12T14:55:57.909123+0000","last_undegraded":"2023-09-12T14:55:57.909123+0000","last_fullsized":"2023-09-12T14:55:57.909123+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:32:13.749664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.192403+0000","last_change":"2023-09-12T14:55:56.941023+0000","last_active":"2023-09-12T14:55:59.192403+0000","last_peered":"2023-09-12T14:55:59.192403+0000","last_clean":"2023-09-12T14:55:59.192403+0000","last_became_active":"2023-09-12T14:55:56.940523+0000","last_became_peered":"2023-09-12T14:55:56.940523+0000","last_unstale":"2023-09-12T14:55:59.192403+0000","last_undegraded":"2023-09-12T14:55:59.192403+0000","last_fullsized":"2023-09-12T14:55:59.192403+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:35:17.945837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.193633+0000","last_change":"2023-09-12T14:55:56.954316+0000","last_active":"2023-09-12T14:55:59.193633+0000","last_peered":"2023-09-12T14:55:59.193633+0000","last_clean":"2023-09-12T14:55:59.193633+0000","last_became_active":"2023-09-12T14:55:56.953782+0000","last_became_peered":"2023-09-12T14:55:56.953782+0000","last_unstale":"2023-09-12T14:55:59.193633+0000","last_undegraded":"2023-09-12T14:55:59.193633+0000","last_fullsized":"2023-09-12T14:55:59.193633+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T21:26:56.144483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.192198+0000","last_change":"2023-09-12T14:55:56.950280+0000","last_active":"2023-09-12T14:55:59.192198+0000","last_peered":"2023-09-12T14:55:59.192198+0000","last_clean":"2023-09-12T14:55:59.192198+0000","last_became_active":"2023-09-12T14:55:56.949563+0000","last_became_peered":"2023-09-12T14:55:56.949563+0000","last_unstale":"2023-09-12T14:55:59.192198+0000","last_undegraded":"2023-09-12T14:55:59.192198+0000","last_fullsized":"2023-09-12T14:55:59.192198+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:26:20.168808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.911721+0000","last_change":"2023-09-12T14:55:56.936036+0000","last_active":"2023-09-12T14:55:57.911721+0000","last_peered":"2023-09-12T14:55:57.911721+0000","last_clean":"2023-09-12T14:55:57.911721+0000","last_became_active":"2023-09-12T14:55:56.935230+0000","last_became_peered":"2023-09-12T14:55:56.935230+0000","last_unstale":"2023-09-12T14:55:57.911721+0000","last_undegraded":"2023-09-12T14:55:57.911721+0000","last_fullsized":"2023-09-12T14:55:57.911721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:53:01.273053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.910903+0000","last_change":"2023-09-12T14:55:56.939673+0000","last_active":"2023-09-12T14:55:57.910903+0000","last_peered":"2023-09-12T14:55:57.910903+0000","last_clean":"2023-09-12T14:55:57.910903+0000","last_became_active":"2023-09-12T14:55:56.938856+0000","last_became_peered":"2023-09-12T14:55:56.938856+0000","last_unstale":"2023-09-12T14:55:57.910903+0000","last_undegraded":"2023-09-12T14:55:57.910903+0000","last_fullsized":"2023-09-12T14:55:57.910903+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:22:06.073591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189711+0000","last_change":"2023-09-12T14:55:56.959107+0000","last_active":"2023-09-12T14:55:59.189711+0000","last_peered":"2023-09-12T14:55:59.189711+0000","last_clean":"2023-09-12T14:55:59.189711+0000","last_became_active":"2023-09-12T14:55:56.957834+0000","last_became_peered":"2023-09-12T14:55:56.957834+0000","last_unstale":"2023-09-12T14:55:59.189711+0000","last_undegraded":"2023-09-12T14:55:59.189711+0000","last_fullsized":"2023-09-12T14:55:59.189711+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:35:17.945837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:58.233870+0000","last_change":"2023-09-12T14:55:56.942862+0000","last_active":"2023-09-12T14:55:58.233870+0000","last_peered":"2023-09-12T14:55:58.233870+0000","last_clean":"2023-09-12T14:55:58.233870+0000","last_became_active":"2023-09-12T14:55:56.941962+0000","last_became_peered":"2023-09-12T14:55:56.941962+0000","last_unstale":"2023-09-12T14:55:58.233870+0000","last_undegraded":"2023-09-12T14:55:58.233870+0000","last_fullsized":"2023-09-12T14:55:58.233870+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:50:12.068597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189026+0000","last_change":"2023-09-12T14:55:56.960407+0000","last_active":"2023-09-12T14:55:59.189026+0000","last_peered":"2023-09-12T14:55:59.189026+0000","last_clean":"2023-09-12T14:55:59.189026+0000","last_became_active":"2023-09-12T14:55:56.959786+0000","last_became_peered":"2023-09-12T14:55:56.959786+0000","last_unstale":"2023-09-12T14:55:59.189026+0000","last_undegraded":"2023-09-12T14:55:59.189026+0000","last_fullsized":"2023-09-12T14:55:59.189026+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:51:57.331781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.903349+0000","last_change":"2023-09-12T14:55:56.954445+0000","last_active":"2023-09-12T14:55:57.903349+0000","last_peered":"2023-09-12T14:55:57.903349+0000","last_clean":"2023-09-12T14:55:57.903349+0000","last_became_active":"2023-09-12T14:55:56.953823+0000","last_became_peered":"2023-09-12T14:55:56.953823+0000","last_unstale":"2023-09-12T14:55:57.903349+0000","last_undegraded":"2023-09-12T14:55:57.903349+0000","last_fullsized":"2023-09-12T14:55:57.903349+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:48:44.893608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.529461+0000","last_change":"2023-09-12T14:55:56.539868+0000","last_active":"2023-09-12T14:55:57.529461+0000","last_peered":"2023-09-12T14:55:57.529461+0000","last_clean":"2023-09-12T14:55:57.529461+0000","last_became_active":"2023-09-12T14:55:56.539227+0000","last_became_peered":"2023-09-12T14:55:56.539227+0000","last_unstale":"2023-09-12T14:55:57.529461+0000","last_undegraded":"2023-09-12T14:55:57.529461+0000","last_fullsized":"2023-09-12T14:55:57.529461+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:44:26.291246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:58.234133+0000","last_change":"2023-09-12T14:55:56.956878+0000","last_active":"2023-09-12T14:55:58.234133+0000","last_peered":"2023-09-12T14:55:58.234133+0000","last_clean":"2023-09-12T14:55:58.234133+0000","last_became_active":"2023-09-12T14:55:56.956210+0000","last_became_peered":"2023-09-12T14:55:56.956210+0000","last_unstale":"2023-09-12T14:55:58.234133+0000","last_undegraded":"2023-09-12T14:55:58.234133+0000","last_fullsized":"2023-09-12T14:55:58.234133+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:22:06.073591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.529479+0000","last_change":"2023-09-12T14:55:56.549272+0000","last_active":"2023-09-12T14:55:57.529479+0000","last_peered":"2023-09-12T14:55:57.529479+0000","last_clean":"2023-09-12T14:55:57.529479+0000","last_became_active":"2023-09-12T14:55:56.548890+0000","last_became_peered":"2023-09-12T14:55:56.548890+0000","last_unstale":"2023-09-12T14:55:57.529479+0000","last_undegraded":"2023-09-12T14:55:57.529479+0000","last_fullsized":"2023-09-12T14:55:57.529479+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:47:23.257700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.910356+0000","last_change":"2023-09-12T14:55:56.940012+0000","last_active":"2023-09-12T14:55:57.910356+0000","last_peered":"2023-09-12T14:55:57.910356+0000","last_clean":"2023-09-12T14:55:57.910356+0000","last_became_active":"2023-09-12T14:55:56.938537+0000","last_became_peered":"2023-09-12T14:55:56.938537+0000","last_unstale":"2023-09-12T14:55:57.910356+0000","last_undegraded":"2023-09-12T14:55:57.910356+0000","last_fullsized":"2023-09-12T14:55:57.910356+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:48:44.893608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.530119+0000","last_change":"2023-09-12T14:55:56.534734+0000","last_active":"2023-09-12T14:55:57.530119+0000","last_peered":"2023-09-12T14:55:57.530119+0000","last_clean":"2023-09-12T14:55:57.530119+0000","last_became_active":"2023-09-12T14:55:56.534334+0000","last_became_peered":"2023-09-12T14:55:56.534334+0000","last_unstale":"2023-09-12T14:55:57.530119+0000","last_undegraded":"2023-09-12T14:55:57.530119+0000","last_fullsized":"2023-09-12T14:55:57.530119+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:05:46.802102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.192370+0000","last_change":"2023-09-12T14:55:56.950640+0000","last_active":"2023-09-12T14:55:59.192370+0000","last_peered":"2023-09-12T14:55:59.192370+0000","last_clean":"2023-09-12T14:55:59.192370+0000","last_became_active":"2023-09-12T14:55:56.949846+0000","last_became_peered":"2023-09-12T14:55:56.949846+0000","last_unstale":"2023-09-12T14:55:59.192370+0000","last_undegraded":"2023-09-12T14:55:59.192370+0000","last_fullsized":"2023-09-12T14:55:59.192370+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:24:20.331640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.525109+0000","last_change":"2023-09-12T14:55:56.536652+0000","last_active":"2023-09-12T14:55:57.525109+0000","last_peered":"2023-09-12T14:55:57.525109+0000","last_clean":"2023-09-12T14:55:57.525109+0000","last_became_active":"2023-09-12T14:55:56.536281+0000","last_became_peered":"2023-09-12T14:55:56.536281+0000","last_unstale":"2023-09-12T14:55:57.525109+0000","last_undegraded":"2023-09-12T14:55:57.525109+0000","last_fullsized":"2023-09-12T14:55:57.525109+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:54:13.129205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189466+0000","last_change":"2023-09-12T14:55:56.961308+0000","last_active":"2023-09-12T14:55:59.189466+0000","last_peered":"2023-09-12T14:55:59.189466+0000","last_clean":"2023-09-12T14:55:59.189466+0000","last_became_active":"2023-09-12T14:55:56.960758+0000","last_became_peered":"2023-09-12T14:55:56.960758+0000","last_unstale":"2023-09-12T14:55:59.189466+0000","last_undegraded":"2023-09-12T14:55:59.189466+0000","last_fullsized":"2023-09-12T14:55:59.189466+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T19:32:13.749664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.528783+0000","last_change":"2023-09-12T14:55:56.544207+0000","last_active":"2023-09-12T14:55:57.528783+0000","last_peered":"2023-09-12T14:55:57.528783+0000","last_clean":"2023-09-12T14:55:57.528783+0000","last_became_active":"2023-09-12T14:55:56.543652+0000","last_became_peered":"2023-09-12T14:55:56.543652+0000","last_unstale":"2023-09-12T14:55:57.528783+0000","last_undegraded":"2023-09-12T14:55:57.528783+0000","last_fullsized":"2023-09-12T14:55:57.528783+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:53:40.126003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.911017+0000","last_change":"2023-09-12T14:55:56.935797+0000","last_active":"2023-09-12T14:55:57.911017+0000","last_peered":"2023-09-12T14:55:57.911017+0000","last_clean":"2023-09-12T14:55:57.911017+0000","last_became_active":"2023-09-12T14:55:56.934895+0000","last_became_peered":"2023-09-12T14:55:56.934895+0000","last_unstale":"2023-09-12T14:55:57.911017+0000","last_undegraded":"2023-09-12T14:55:57.911017+0000","last_fullsized":"2023-09-12T14:55:57.911017+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:26:20.168808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.528262+0000","last_change":"2023-09-12T14:55:56.541914+0000","last_active":"2023-09-12T14:55:57.528262+0000","last_peered":"2023-09-12T14:55:57.528262+0000","last_clean":"2023-09-12T14:55:57.528262+0000","last_became_active":"2023-09-12T14:55:56.541110+0000","last_became_peered":"2023-09-12T14:55:56.541110+0000","last_unstale":"2023-09-12T14:55:57.528262+0000","last_undegraded":"2023-09-12T14:55:57.528262+0000","last_fullsized":"2023-09-12T14:55:57.528262+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:27:22.656167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.909839+0000","last_change":"2023-09-12T14:55:56.944913+0000","last_active":"2023-09-12T14:55:57.909839+0000","last_peered":"2023-09-12T14:55:57.909839+0000","last_clean":"2023-09-12T14:55:57.909839+0000","last_became_active":"2023-09-12T14:55:56.942134+0000","last_became_peered":"2023-09-12T14:55:56.942134+0000","last_unstale":"2023-09-12T14:55:57.909839+0000","last_undegraded":"2023-09-12T14:55:57.909839+0000","last_fullsized":"2023-09-12T14:55:57.909839+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:50:32.960307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.903470+0000","last_change":"2023-09-12T14:55:56.531053+0000","last_active":"2023-09-12T14:55:56.903470+0000","last_peered":"2023-09-12T14:55:56.903470+0000","last_clean":"2023-09-12T14:55:56.903470+0000","last_became_active":"2023-09-12T14:55:56.530748+0000","last_became_peered":"2023-09-12T14:55:56.530748+0000","last_unstale":"2023-09-12T14:55:56.903470+0000","last_undegraded":"2023-09-12T14:55:56.903470+0000","last_fullsized":"2023-09-12T14:55:56.903470+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:50:31.955735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189427+0000","last_change":"2023-09-12T14:55:56.940009+0000","last_active":"2023-09-12T14:55:59.189427+0000","last_peered":"2023-09-12T14:55:59.189427+0000","last_clean":"2023-09-12T14:55:59.189427+0000","last_became_active":"2023-09-12T14:55:56.939402+0000","last_became_peered":"2023-09-12T14:55:56.939402+0000","last_unstale":"2023-09-12T14:55:59.189427+0000","last_undegraded":"2023-09-12T14:55:59.189427+0000","last_fullsized":"2023-09-12T14:55:59.189427+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:10:13.906110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.913244+0000","last_change":"2023-09-12T14:55:56.541874+0000","last_active":"2023-09-12T14:55:56.913244+0000","last_peered":"2023-09-12T14:55:56.913244+0000","last_clean":"2023-09-12T14:55:56.913244+0000","last_became_active":"2023-09-12T14:55:56.539725+0000","last_became_peered":"2023-09-12T14:55:56.539725+0000","last_unstale":"2023-09-12T14:55:56.913244+0000","last_undegraded":"2023-09-12T14:55:56.913244+0000","last_fullsized":"2023-09-12T14:55:56.913244+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:54:13.129205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.911751+0000","last_change":"2023-09-12T14:55:56.935986+0000","last_active":"2023-09-12T14:55:57.911751+0000","last_peered":"2023-09-12T14:55:57.911751+0000","last_clean":"2023-09-12T14:55:57.911751+0000","last_became_active":"2023-09-12T14:55:56.935145+0000","last_became_peered":"2023-09-12T14:55:56.935145+0000","last_unstale":"2023-09-12T14:55:57.911751+0000","last_undegraded":"2023-09-12T14:55:57.911751+0000","last_fullsized":"2023-09-12T14:55:57.911751+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:15:17.791216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.906351+0000","last_change":"2023-09-12T14:55:56.549758+0000","last_active":"2023-09-12T14:55:56.906351+0000","last_peered":"2023-09-12T14:55:56.906351+0000","last_clean":"2023-09-12T14:55:56.906351+0000","last_became_active":"2023-09-12T14:55:56.549407+0000","last_became_peered":"2023-09-12T14:55:56.549407+0000","last_unstale":"2023-09-12T14:55:56.906351+0000","last_undegraded":"2023-09-12T14:55:56.906351+0000","last_fullsized":"2023-09-12T14:55:56.906351+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:51:56.327209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.193650+0000","last_change":"2023-09-12T14:55:56.943011+0000","last_active":"2023-09-12T14:55:59.193650+0000","last_peered":"2023-09-12T14:55:59.193650+0000","last_clean":"2023-09-12T14:55:59.193650+0000","last_became_active":"2023-09-12T14:55:56.942313+0000","last_became_peered":"2023-09-12T14:55:56.942313+0000","last_unstale":"2023-09-12T14:55:59.193650+0000","last_undegraded":"2023-09-12T14:55:59.193650+0000","last_fullsized":"2023-09-12T14:55:59.193650+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T16:00:06.884652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.918809+0000","last_change":"2023-09-12T14:55:56.532566+0000","last_active":"2023-09-12T14:55:56.918809+0000","last_peered":"2023-09-12T14:55:56.918809+0000","last_clean":"2023-09-12T14:55:56.918809+0000","last_became_active":"2023-09-12T14:55:56.532122+0000","last_became_peered":"2023-09-12T14:55:56.532122+0000","last_unstale":"2023-09-12T14:55:56.918809+0000","last_undegraded":"2023-09-12T14:55:56.918809+0000","last_fullsized":"2023-09-12T14:55:56.918809+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:46:32.695633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.191443+0000","last_change":"2023-09-12T14:55:56.946251+0000","last_active":"2023-09-12T14:55:59.191443+0000","last_peered":"2023-09-12T14:55:59.191443+0000","last_clean":"2023-09-12T14:55:59.191443+0000","last_became_active":"2023-09-12T14:55:56.945833+0000","last_became_peered":"2023-09-12T14:55:56.945833+0000","last_unstale":"2023-09-12T14:55:59.191443+0000","last_undegraded":"2023-09-12T14:55:59.191443+0000","last_fullsized":"2023-09-12T14:55:59.191443+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:15:17.791216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.530322+0000","last_change":"2023-09-12T14:55:56.550717+0000","last_active":"2023-09-12T14:55:57.530322+0000","last_peered":"2023-09-12T14:55:57.530322+0000","last_clean":"2023-09-12T14:55:57.530322+0000","last_became_active":"2023-09-12T14:55:56.550055+0000","last_became_peered":"2023-09-12T14:55:56.550055+0000","last_unstale":"2023-09-12T14:55:57.530322+0000","last_undegraded":"2023-09-12T14:55:57.530322+0000","last_fullsized":"2023-09-12T14:55:57.530322+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:51:56.327209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.911356+0000","last_change":"2023-09-12T14:55:56.944565+0000","last_active":"2023-09-12T14:55:57.911356+0000","last_peered":"2023-09-12T14:55:57.911356+0000","last_clean":"2023-09-12T14:55:57.911356+0000","last_became_active":"2023-09-12T14:55:56.943807+0000","last_became_peered":"2023-09-12T14:55:56.943807+0000","last_unstale":"2023-09-12T14:55:57.911356+0000","last_undegraded":"2023-09-12T14:55:57.911356+0000","last_fullsized":"2023-09-12T14:55:57.911356+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:24:20.331640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.532314+0000","last_change":"2023-09-12T14:55:56.534785+0000","last_active":"2023-09-12T14:55:57.532314+0000","last_peered":"2023-09-12T14:55:57.532314+0000","last_clean":"2023-09-12T14:55:57.532314+0000","last_became_active":"2023-09-12T14:55:56.534401+0000","last_became_peered":"2023-09-12T14:55:56.534401+0000","last_unstale":"2023-09-12T14:55:57.532314+0000","last_undegraded":"2023-09-12T14:55:57.532314+0000","last_fullsized":"2023-09-12T14:55:57.532314+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:08:55.579592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.190836+0000","last_change":"2023-09-12T14:55:56.936770+0000","last_active":"2023-09-12T14:55:59.190836+0000","last_peered":"2023-09-12T14:55:59.190836+0000","last_clean":"2023-09-12T14:55:59.190836+0000","last_became_active":"2023-09-12T14:55:56.936011+0000","last_became_peered":"2023-09-12T14:55:56.936011+0000","last_unstale":"2023-09-12T14:55:59.190836+0000","last_undegraded":"2023-09-12T14:55:59.190836+0000","last_fullsized":"2023-09-12T14:55:59.190836+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:51:57.331781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.529740+0000","last_change":"2023-09-12T14:55:56.542724+0000","last_active":"2023-09-12T14:55:57.529740+0000","last_peered":"2023-09-12T14:55:57.529740+0000","last_clean":"2023-09-12T14:55:57.529740+0000","last_became_active":"2023-09-12T14:55:56.541958+0000","last_became_peered":"2023-09-12T14:55:56.541958+0000","last_unstale":"2023-09-12T14:55:57.529740+0000","last_undegraded":"2023-09-12T14:55:57.529740+0000","last_fullsized":"2023-09-12T14:55:57.529740+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:48:34.208772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.191555+0000","last_change":"2023-09-12T14:55:56.927415+0000","last_active":"2023-09-12T14:55:59.191555+0000","last_peered":"2023-09-12T14:55:59.191555+0000","last_clean":"2023-09-12T14:55:59.191555+0000","last_became_active":"2023-09-12T14:55:56.926767+0000","last_became_peered":"2023-09-12T14:55:56.926767+0000","last_unstale":"2023-09-12T14:55:59.191555+0000","last_undegraded":"2023-09-12T14:55:59.191555+0000","last_fullsized":"2023-09-12T14:55:59.191555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:51:11.665671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.915603+0000","last_change":"2023-09-12T14:55:56.533893+0000","last_active":"2023-09-12T14:55:56.915603+0000","last_peered":"2023-09-12T14:55:56.915603+0000","last_clean":"2023-09-12T14:55:56.915603+0000","last_became_active":"2023-09-12T14:55:56.533521+0000","last_became_peered":"2023-09-12T14:55:56.533521+0000","last_unstale":"2023-09-12T14:55:56.915603+0000","last_undegraded":"2023-09-12T14:55:56.915603+0000","last_fullsized":"2023-09-12T14:55:56.915603+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:50:31.955735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189448+0000","last_change":"2023-09-12T14:55:56.961621+0000","last_active":"2023-09-12T14:55:59.189448+0000","last_peered":"2023-09-12T14:55:59.189448+0000","last_clean":"2023-09-12T14:55:59.189448+0000","last_became_active":"2023-09-12T14:55:56.960906+0000","last_became_peered":"2023-09-12T14:55:56.960906+0000","last_unstale":"2023-09-12T14:55:59.189448+0000","last_undegraded":"2023-09-12T14:55:59.189448+0000","last_fullsized":"2023-09-12T14:55:59.189448+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:08:56.584164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.903112+0000","last_change":"2023-09-12T14:55:56.549738+0000","last_active":"2023-09-12T14:55:56.903112+0000","last_peered":"2023-09-12T14:55:56.903112+0000","last_clean":"2023-09-12T14:55:56.903112+0000","last_became_active":"2023-09-12T14:55:56.549335+0000","last_became_peered":"2023-09-12T14:55:56.549335+0000","last_unstale":"2023-09-12T14:55:56.903112+0000","last_undegraded":"2023-09-12T14:55:56.903112+0000","last_fullsized":"2023-09-12T14:55:56.903112+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:44:26.291246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:58.233939+0000","last_change":"2023-09-12T14:55:56.951267+0000","last_active":"2023-09-12T14:55:58.233939+0000","last_peered":"2023-09-12T14:55:58.233939+0000","last_clean":"2023-09-12T14:55:58.233939+0000","last_became_active":"2023-09-12T14:55:56.949172+0000","last_became_peered":"2023-09-12T14:55:56.949172+0000","last_unstale":"2023-09-12T14:55:58.233939+0000","last_undegraded":"2023-09-12T14:55:58.233939+0000","last_fullsized":"2023-09-12T14:55:58.233939+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:48:44.893608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.530290+0000","last_change":"2023-09-12T14:55:56.534769+0000","last_active":"2023-09-12T14:55:57.530290+0000","last_peered":"2023-09-12T14:55:57.530290+0000","last_clean":"2023-09-12T14:55:57.530290+0000","last_became_active":"2023-09-12T14:55:56.534371+0000","last_became_peered":"2023-09-12T14:55:56.534371+0000","last_unstale":"2023-09-12T14:55:57.530290+0000","last_undegraded":"2023-09-12T14:55:57.530290+0000","last_fullsized":"2023-09-12T14:55:57.530290+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:24:04.838737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.910240+0000","last_change":"2023-09-12T14:55:56.948577+0000","last_active":"2023-09-12T14:55:57.910240+0000","last_peered":"2023-09-12T14:55:57.910240+0000","last_clean":"2023-09-12T14:55:57.910240+0000","last_became_active":"2023-09-12T14:55:56.946473+0000","last_became_peered":"2023-09-12T14:55:56.946473+0000","last_unstale":"2023-09-12T14:55:57.910240+0000","last_undegraded":"2023-09-12T14:55:57.910240+0000","last_fullsized":"2023-09-12T14:55:57.910240+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:50:12.068597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.525763+0000","last_change":"2023-09-12T14:55:56.532423+0000","last_active":"2023-09-12T14:55:57.525763+0000","last_peered":"2023-09-12T14:55:57.525763+0000","last_clean":"2023-09-12T14:55:57.525763+0000","last_became_active":"2023-09-12T14:55:56.531971+0000","last_became_peered":"2023-09-12T14:55:56.531971+0000","last_unstale":"2023-09-12T14:55:57.525763+0000","last_undegraded":"2023-09-12T14:55:57.525763+0000","last_fullsized":"2023-09-12T14:55:57.525763+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:24:52.205479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.911747+0000","last_change":"2023-09-12T14:55:56.948863+0000","last_active":"2023-09-12T14:55:57.911747+0000","last_peered":"2023-09-12T14:55:57.911747+0000","last_clean":"2023-09-12T14:55:57.911747+0000","last_became_active":"2023-09-12T14:55:56.947965+0000","last_became_peered":"2023-09-12T14:55:56.947965+0000","last_unstale":"2023-09-12T14:55:57.911747+0000","last_undegraded":"2023-09-12T14:55:57.911747+0000","last_fullsized":"2023-09-12T14:55:57.911747+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:54:34.731553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.528557+0000","last_change":"2023-09-12T14:55:56.538492+0000","last_active":"2023-09-12T14:55:57.528557+0000","last_peered":"2023-09-12T14:55:57.528557+0000","last_clean":"2023-09-12T14:55:57.528557+0000","last_became_active":"2023-09-12T14:55:56.537724+0000","last_became_peered":"2023-09-12T14:55:56.537724+0000","last_unstale":"2023-09-12T14:55:57.528557+0000","last_undegraded":"2023-09-12T14:55:57.528557+0000","last_fullsized":"2023-09-12T14:55:57.528557+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:47:23.257700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.192004+0000","last_change":"2023-09-12T14:55:56.936455+0000","last_active":"2023-09-12T14:55:59.192004+0000","last_peered":"2023-09-12T14:55:59.192004+0000","last_clean":"2023-09-12T14:55:59.192004+0000","last_became_active":"2023-09-12T14:55:56.935777+0000","last_became_peered":"2023-09-12T14:55:56.935777+0000","last_unstale":"2023-09-12T14:55:59.192004+0000","last_undegraded":"2023-09-12T14:55:59.192004+0000","last_fullsized":"2023-09-12T14:55:59.192004+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:37:00.305524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.528588+0000","last_change":"2023-09-12T14:55:56.539779+0000","last_active":"2023-09-12T14:55:57.528588+0000","last_peered":"2023-09-12T14:55:57.528588+0000","last_clean":"2023-09-12T14:55:57.528588+0000","last_became_active":"2023-09-12T14:55:56.539014+0000","last_became_peered":"2023-09-12T14:55:56.539014+0000","last_unstale":"2023-09-12T14:55:57.528588+0000","last_undegraded":"2023-09-12T14:55:57.528588+0000","last_fullsized":"2023-09-12T14:55:57.528588+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:31:39.967705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.190689+0000","last_change":"2023-09-12T14:55:56.942708+0000","last_active":"2023-09-12T14:55:59.190689+0000","last_peered":"2023-09-12T14:55:59.190689+0000","last_clean":"2023-09-12T14:55:59.190689+0000","last_became_active":"2023-09-12T14:55:56.941801+0000","last_became_peered":"2023-09-12T14:55:56.941801+0000","last_unstale":"2023-09-12T14:55:59.190689+0000","last_undegraded":"2023-09-12T14:55:59.190689+0000","last_fullsized":"2023-09-12T14:55:59.190689+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:24:05.311518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.919648+0000","last_change":"2023-09-12T14:55:56.542930+0000","last_active":"2023-09-12T14:55:56.919648+0000","last_peered":"2023-09-12T14:55:56.919648+0000","last_clean":"2023-09-12T14:55:56.919648+0000","last_became_active":"2023-09-12T14:55:56.541322+0000","last_became_peered":"2023-09-12T14:55:56.541322+0000","last_unstale":"2023-09-12T14:55:56.919648+0000","last_undegraded":"2023-09-12T14:55:56.919648+0000","last_fullsized":"2023-09-12T14:55:56.919648+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:46:32.695633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189301+0000","last_change":"2023-09-12T14:55:56.945804+0000","last_active":"2023-09-12T14:55:59.189301+0000","last_peered":"2023-09-12T14:55:59.189301+0000","last_clean":"2023-09-12T14:55:59.189301+0000","last_became_active":"2023-09-12T14:55:56.945486+0000","last_became_peered":"2023-09-12T14:55:56.945486+0000","last_unstale":"2023-09-12T14:55:59.189301+0000","last_undegraded":"2023-09-12T14:55:59.189301+0000","last_fullsized":"2023-09-12T14:55:59.189301+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:53:41.130575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.528421+0000","last_change":"2023-09-12T14:55:56.520900+0000","last_active":"2023-09-12T14:55:57.528421+0000","last_peered":"2023-09-12T14:55:57.528421+0000","last_clean":"2023-09-12T14:55:57.528421+0000","last_became_active":"2023-09-12T14:55:56.515607+0000","last_became_peered":"2023-09-12T14:55:56.515607+0000","last_unstale":"2023-09-12T14:55:57.528421+0000","last_undegraded":"2023-09-12T14:55:57.528421+0000","last_fullsized":"2023-09-12T14:55:57.528421+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:44:26.291246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.190265+0000","last_change":"2023-09-12T14:55:56.924355+0000","last_active":"2023-09-12T14:55:59.190265+0000","last_peered":"2023-09-12T14:55:59.190265+0000","last_clean":"2023-09-12T14:55:59.190265+0000","last_became_active":"2023-09-12T14:55:56.924044+0000","last_became_peered":"2023-09-12T14:55:56.924044+0000","last_unstale":"2023-09-12T14:55:59.190265+0000","last_undegraded":"2023-09-12T14:55:59.190265+0000","last_fullsized":"2023-09-12T14:55:59.190265+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:30:21.532796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.929588+0000","last_change":"2023-09-12T14:55:56.538763+0000","last_active":"2023-09-12T14:55:56.929588+0000","last_peered":"2023-09-12T14:55:56.929588+0000","last_clean":"2023-09-12T14:55:56.929588+0000","last_became_active":"2023-09-12T14:55:56.538095+0000","last_became_peered":"2023-09-12T14:55:56.538095+0000","last_unstale":"2023-09-12T14:55:56.929588+0000","last_undegraded":"2023-09-12T14:55:56.929588+0000","last_fullsized":"2023-09-12T14:55:56.929588+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:27:22.656167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189875+0000","last_change":"2023-09-12T14:55:56.925305+0000","last_active":"2023-09-12T14:55:59.189875+0000","last_peered":"2023-09-12T14:55:59.189875+0000","last_clean":"2023-09-12T14:55:59.189875+0000","last_became_active":"2023-09-12T14:55:56.924770+0000","last_became_peered":"2023-09-12T14:55:56.924770+0000","last_unstale":"2023-09-12T14:55:59.189875+0000","last_undegraded":"2023-09-12T14:55:59.189875+0000","last_fullsized":"2023-09-12T14:55:59.189875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:48:35.213344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.913726+0000","last_change":"2023-09-12T14:55:56.543061+0000","last_active":"2023-09-12T14:55:56.913726+0000","last_peered":"2023-09-12T14:55:56.913726+0000","last_clean":"2023-09-12T14:55:56.913726+0000","last_became_active":"2023-09-12T14:55:56.542293+0000","last_became_peered":"2023-09-12T14:55:56.542293+0000","last_unstale":"2023-09-12T14:55:56.913726+0000","last_undegraded":"2023-09-12T14:55:56.913726+0000","last_fullsized":"2023-09-12T14:55:56.913726+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:24:36.430305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189688+0000","last_change":"2023-09-12T14:55:56.958745+0000","last_active":"2023-09-12T14:55:59.189688+0000","last_peered":"2023-09-12T14:55:59.189688+0000","last_clean":"2023-09-12T14:55:59.189688+0000","last_became_active":"2023-09-12T14:55:56.957778+0000","last_became_peered":"2023-09-12T14:55:56.957778+0000","last_unstale":"2023-09-12T14:55:59.189688+0000","last_undegraded":"2023-09-12T14:55:59.189688+0000","last_fullsized":"2023-09-12T14:55:59.189688+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:24:20.331640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.909971+0000","last_change":"2023-09-12T14:55:56.550760+0000","last_active":"2023-09-12T14:55:56.909971+0000","last_peered":"2023-09-12T14:55:56.909971+0000","last_clean":"2023-09-12T14:55:56.909971+0000","last_became_active":"2023-09-12T14:55:56.550171+0000","last_became_peered":"2023-09-12T14:55:56.550171+0000","last_unstale":"2023-09-12T14:55:56.909971+0000","last_undegraded":"2023-09-12T14:55:56.909971+0000","last_fullsized":"2023-09-12T14:55:56.909971+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:50:11.064026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.190810+0000","last_change":"2023-09-12T14:55:56.938612+0000","last_active":"2023-09-12T14:55:59.190810+0000","last_peered":"2023-09-12T14:55:59.190810+0000","last_clean":"2023-09-12T14:55:59.190810+0000","last_became_active":"2023-09-12T14:55:56.936787+0000","last_became_peered":"2023-09-12T14:55:56.936787+0000","last_unstale":"2023-09-12T14:55:59.190810+0000","last_undegraded":"2023-09-12T14:55:59.190810+0000","last_fullsized":"2023-09-12T14:55:59.190810+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:47:24.262272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.524768+0000","last_change":"2023-09-12T14:55:56.526327+0000","last_active":"2023-09-12T14:55:57.524768+0000","last_peered":"2023-09-12T14:55:57.524768+0000","last_clean":"2023-09-12T14:55:57.524768+0000","last_became_active":"2023-09-12T14:55:56.525989+0000","last_became_peered":"2023-09-12T14:55:56.525989+0000","last_unstale":"2023-09-12T14:55:57.524768+0000","last_undegraded":"2023-09-12T14:55:57.524768+0000","last_fullsized":"2023-09-12T14:55:57.524768+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:05:46.802102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.911917+0000","last_change":"2023-09-12T14:55:56.943720+0000","last_active":"2023-09-12T14:55:57.911917+0000","last_peered":"2023-09-12T14:55:57.911917+0000","last_clean":"2023-09-12T14:55:57.911917+0000","last_became_active":"2023-09-12T14:55:56.942886+0000","last_became_peered":"2023-09-12T14:55:56.942886+0000","last_unstale":"2023-09-12T14:55:57.911917+0000","last_undegraded":"2023-09-12T14:55:57.911917+0000","last_fullsized":"2023-09-12T14:55:57.911917+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T01:51:11.665671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.903471+0000","last_change":"2023-09-12T14:55:56.528396+0000","last_active":"2023-09-12T14:55:57.903471+0000","last_peered":"2023-09-12T14:55:57.903471+0000","last_clean":"2023-09-12T14:55:57.903471+0000","last_became_active":"2023-09-12T14:55:56.527993+0000","last_became_peered":"2023-09-12T14:55:56.527993+0000","last_unstale":"2023-09-12T14:55:57.903471+0000","last_undegraded":"2023-09-12T14:55:57.903471+0000","last_fullsized":"2023-09-12T14:55:57.903471+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:27:22.656167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.903268+0000","last_change":"2023-09-12T14:55:56.938713+0000","last_active":"2023-09-12T14:55:57.903268+0000","last_peered":"2023-09-12T14:55:57.903268+0000","last_clean":"2023-09-12T14:55:57.903268+0000","last_became_active":"2023-09-12T14:55:56.937836+0000","last_became_peered":"2023-09-12T14:55:56.937836+0000","last_unstale":"2023-09-12T14:55:57.903268+0000","last_undegraded":"2023-09-12T14:55:57.903268+0000","last_fullsized":"2023-09-12T14:55:57.903268+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:50:12.068597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.915820+0000","last_change":"2023-09-12T14:55:56.531043+0000","last_active":"2023-09-12T14:55:56.915820+0000","last_peered":"2023-09-12T14:55:56.915820+0000","last_clean":"2023-09-12T14:55:56.915820+0000","last_became_active":"2023-09-12T14:55:56.530557+0000","last_became_peered":"2023-09-12T14:55:56.530557+0000","last_unstale":"2023-09-12T14:55:56.915820+0000","last_undegraded":"2023-09-12T14:55:56.915820+0000","last_fullsized":"2023-09-12T14:55:56.915820+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T02:50:11.064026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:58.233954+0000","last_change":"2023-09-12T14:55:56.944558+0000","last_active":"2023-09-12T14:55:58.233954+0000","last_peered":"2023-09-12T14:55:58.233954+0000","last_clean":"2023-09-12T14:55:58.233954+0000","last_became_active":"2023-09-12T14:55:56.943774+0000","last_became_peered":"2023-09-12T14:55:56.943774+0000","last_unstale":"2023-09-12T14:55:58.233954+0000","last_undegraded":"2023-09-12T14:55:58.233954+0000","last_fullsized":"2023-09-12T14:55:58.233954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:24:53.210051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.922923+0000","last_change":"2023-09-12T14:55:56.536269+0000","last_active":"2023-09-12T14:55:56.922923+0000","last_peered":"2023-09-12T14:55:56.922923+0000","last_clean":"2023-09-12T14:55:56.922923+0000","last_became_active":"2023-09-12T14:55:56.535875+0000","last_became_peered":"2023-09-12T14:55:56.535875+0000","last_unstale":"2023-09-12T14:55:56.922923+0000","last_undegraded":"2023-09-12T14:55:56.922923+0000","last_fullsized":"2023-09-12T14:55:56.922923+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:51:37.543569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.903377+0000","last_change":"2023-09-12T14:55:56.945018+0000","last_active":"2023-09-12T14:55:57.903377+0000","last_peered":"2023-09-12T14:55:57.903377+0000","last_clean":"2023-09-12T14:55:57.903377+0000","last_became_active":"2023-09-12T14:55:56.944343+0000","last_became_peered":"2023-09-12T14:55:56.944343+0000","last_unstale":"2023-09-12T14:55:57.903377+0000","last_undegraded":"2023-09-12T14:55:57.903377+0000","last_fullsized":"2023-09-12T14:55:57.903377+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T00:24:53.210051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.915674+0000","last_change":"2023-09-12T14:55:56.536191+0000","last_active":"2023-09-12T14:55:56.915674+0000","last_peered":"2023-09-12T14:55:56.915674+0000","last_clean":"2023-09-12T14:55:56.915674+0000","last_became_active":"2023-09-12T14:55:56.535847+0000","last_became_peered":"2023-09-12T14:55:56.535847+0000","last_unstale":"2023-09-12T14:55:56.915674+0000","last_undegraded":"2023-09-12T14:55:56.915674+0000","last_fullsized":"2023-09-12T14:55:56.915674+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T18:24:04.838737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189930+0000","last_change":"2023-09-12T14:55:56.942522+0000","last_active":"2023-09-12T14:55:59.189930+0000","last_peered":"2023-09-12T14:55:59.189930+0000","last_clean":"2023-09-12T14:55:59.189930+0000","last_became_active":"2023-09-12T14:55:56.941639+0000","last_became_peered":"2023-09-12T14:55:56.941639+0000","last_unstale":"2023-09-12T14:55:59.189930+0000","last_undegraded":"2023-09-12T14:55:59.189930+0000","last_fullsized":"2023-09-12T14:55:59.189930+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:54:34.731553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.914061+0000","last_change":"2023-09-12T14:55:56.541270+0000","last_active":"2023-09-12T14:55:56.914061+0000","last_peered":"2023-09-12T14:55:56.914061+0000","last_clean":"2023-09-12T14:55:56.914061+0000","last_became_active":"2023-09-12T14:55:56.540773+0000","last_became_peered":"2023-09-12T14:55:56.540773+0000","last_unstale":"2023-09-12T14:55:56.914061+0000","last_undegraded":"2023-09-12T14:55:56.914061+0000","last_fullsized":"2023-09-12T14:55:56.914061+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T17:15:12.274569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:57.909161+0000","last_change":"2023-09-12T14:55:56.937532+0000","last_active":"2023-09-12T14:55:57.909161+0000","last_peered":"2023-09-12T14:55:57.909161+0000","last_clean":"2023-09-12T14:55:57.909161+0000","last_became_active":"2023-09-12T14:55:56.936799+0000","last_became_peered":"2023-09-12T14:55:56.936799+0000","last_unstale":"2023-09-12T14:55:57.909161+0000","last_undegraded":"2023-09-12T14:55:57.909161+0000","last_fullsized":"2023-09-12T14:55:57.909161+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:51:38.548141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.903377+0000","last_change":"2023-09-12T14:55:56.549806+0000","last_active":"2023-09-12T14:55:56.903377+0000","last_peered":"2023-09-12T14:55:56.903377+0000","last_clean":"2023-09-12T14:55:56.903377+0000","last_became_active":"2023-09-12T14:55:56.549413+0000","last_became_peered":"2023-09-12T14:55:56.549413+0000","last_unstale":"2023-09-12T14:55:56.903377+0000","last_undegraded":"2023-09-12T14:55:56.903377+0000","last_fullsized":"2023-09-12T14:55:56.903377+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:51:37.543569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.189609+0000","last_change":"2023-09-12T14:55:56.954426+0000","last_active":"2023-09-12T14:55:59.189609+0000","last_peered":"2023-09-12T14:55:59.189609+0000","last_clean":"2023-09-12T14:55:59.189609+0000","last_became_active":"2023-09-12T14:55:56.953811+0000","last_became_peered":"2023-09-12T14:55:56.953811+0000","last_unstale":"2023-09-12T14:55:59.189609+0000","last_undegraded":"2023-09-12T14:55:59.189609+0000","last_fullsized":"2023-09-12T14:55:59.189609+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:56.493444+0000","last_clean_scrub_stamp":"2023-09-12T14:55:56.493444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T15:26:20.168808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.929496+0000","last_change":"2023-09-12T14:55:56.545462+0000","last_active":"2023-09-12T14:55:56.929496+0000","last_peered":"2023-09-12T14:55:56.929496+0000","last_clean":"2023-09-12T14:55:56.929496+0000","last_became_active":"2023-09-12T14:55:56.544959+0000","last_became_peered":"2023-09-12T14:55:56.544959+0000","last_unstale":"2023-09-12T14:55:56.929496+0000","last_undegraded":"2023-09-12T14:55:56.929496+0000","last_fullsized":"2023-09-12T14:55:56.929496+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:55:55.488872+0000","last_clean_scrub_stamp":"2023-09-12T14:55:55.488872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T22:46:32.695633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":176,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-09-12T14:55:56.923028+0000","last_change":"2023-09-12T14:54:46.965446+0000","last_active":"2023-09-12T14:55:56.923028+0000","last_peered":"2023-09-12T14:55:56.923028+0000","last_clean":"2023-09-12T14:55:56.923028+0000","last_became_active":"2023-09-12T14:54:46.963236+0000","last_became_peered":"2023-09-12T14:54:46.963236+0000","last_unstale":"2023-09-12T14:55:56.923028+0000","last_undegraded":"2023-09-12T14:55:56.923028+0000","last_fullsized":"2023-09-12T14:55:56.923028+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:45.804232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:45.804232+0000","last_clean_scrub_stamp":"2023-09-12T14:54:45.804232+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:24:45.670513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":737386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.95299999999999996}]},{"osd":1,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":1.119}]},{"osd":2,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.145},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.05}]},{"osd":3,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.82699999999999996}]},{"osd":4,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.071}]},{"osd":5,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.70199999999999996}]},{"osd":6,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.76600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.68500000000000005}]},{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.88600000000000001}]},{"osd":2,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.628},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":1.161}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.052},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.73499999999999999}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":1.2150000000000001}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":1.115},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":1.2969999999999999}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.01},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":1.347}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":1.0680000000000001}]},{"osd":1,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.014}]},{"osd":2,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.222},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.89900000000000002}]},{"osd":3,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":1.1759999999999999},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":1.337}]},{"osd":4,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.3640000000000001}]},{"osd":6,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.516,"5min":2.516,"15min":2.516},"last":1.103},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":1.292}]},{"osd":7,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.248},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":1.391}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":380928,"data_stored":147032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.88400000000000001}]},{"osd":1,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.83499999999999996}]},{"osd":2,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":0.96799999999999997}]},{"osd":3,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":1.0069999999999999}]},{"osd":5,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.1739999999999999},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.94399999999999995}]},{"osd":6,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":1.0880000000000001}]},{"osd":7,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":1.044},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":1.133}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.38},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.46700000000000003}]},{"osd":1,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.436}]},{"osd":2,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.54200000000000004}]},{"osd":4,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.81100000000000005}]},{"osd":5,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.622},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.73199999999999998}]},{"osd":6,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.871},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.66100000000000003}]},{"osd":7,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.67000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":146998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.66500000000000004}]},{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.621},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":1.0549999999999999}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.75900000000000001}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.80700000000000005}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.91200000000000003}]},{"osd":6,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.1989999999999998,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":1.0840000000000001}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":1.02}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.75800000000000001}]},{"osd":2,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.84899999999999998}]},{"osd":3,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.67600000000000005}]},{"osd":4,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":1.3440000000000001},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.179}]},{"osd":5,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":1.292}]},{"osd":6,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":1.208}]},{"osd":7,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":1.254},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.0860000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":962560,"data_stored":736277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.372},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.312}]},{"osd":2,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.38500000000000001}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.505},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.46400000000000002}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.52900000000000003}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":0.59899999999999998},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":0.78800000000000003}]},{"osd":6,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.73799999999999999}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.755},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.92900000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-12T14:56:05.332 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-12T14:56:05.333 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-09-12T14:56:05.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.500+0000 7fdfdedd9640 1 -- 172.21.15.131:0/1169062945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfc40039b0 msgr2=0x7fdfc4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:05.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.500+0000 7fdfdedd9640 1 --2- 172.21.15.131:0/1169062945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfc40039b0 0x7fdfc4003dd0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0009b30 tx=0x7fdfc001d220 comp rx=0 tx=0).stop 2023-09-12T14:56:05.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.500+0000 7fdfdd376640 1 -- 172.21.15.131:0/1169062945 reap_dead start 2023-09-12T14:56:05.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.500+0000 7fdfdedd9640 1 -- 172.21.15.131:0/1169062945 shutdown_connections 2023-09-12T14:56:05.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.500+0000 7fdfdedd9640 1 -- 172.21.15.131:0/1169062945 >> 172.21.15.131:0/1169062945 conn(0x7fdfd8069840 msgr2=0x7fdfd80cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:05.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.500+0000 7fdfdedd9640 1 -- 172.21.15.131:0/1169062945 shutdown_connections 2023-09-12T14:56:05.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.500+0000 7fdfdedd9640 1 -- 172.21.15.131:0/1169062945 wait complete. 2023-09-12T14:56:05.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.500+0000 7fdfdedd9640 1 Processor -- start 2023-09-12T14:56:05.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.500+0000 7fdfdedd9640 1 -- start start 2023-09-12T14:56:05.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.500+0000 7fdfdedd9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfc40039b0 0x7fdfd81503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:05.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.500+0000 7fdfdedd9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd81f13f0 0x7fdfd8152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.500+0000 7fdfdedd9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfd8153250 0x7fdfd81596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.500+0000 7fdfdedd9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd8153700 con 0x7fdfd8153250 2023-09-12T14:56:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.504+0000 7fdfdedd9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd814e5c0 con 0x7fdfd81f13f0 2023-09-12T14:56:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.504+0000 7fdfdedd9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd814e8a0 con 0x7fdfc40039b0 2023-09-12T14:56:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.504+0000 7fdfdd376640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfd8153250 0x7fdfd81596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:05.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.504+0000 7fdfdd376640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfd8153250 0x7fdfd81596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37462/0 (socket says 172.21.15.131:37462) 2023-09-12T14:56:05.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.504+0000 7fdfdd376640 1 -- 172.21.15.131:0/3531590329 learned_addr learned my addr 172.21.15.131:0/3531590329 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:05.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.504+0000 7fdfd7fff640 1 --2- 172.21.15.131:0/3531590329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd81f13f0 0x7fdfd8152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:05.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.504+0000 7fdfdd376640 1 -- 172.21.15.131:0/3531590329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfc40039b0 msgr2=0x7fdfd81503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:05.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.504+0000 7fdfdd376640 1 --2- 172.21.15.131:0/3531590329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfc40039b0 0x7fdfd81503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:05.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.504+0000 7fdfdd376640 1 -- 172.21.15.131:0/3531590329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd81f13f0 msgr2=0x7fdfd8152d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:05.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.504+0000 7fdfdd376640 1 --2- 172.21.15.131:0/3531590329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd81f13f0 0x7fdfd8152d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:05.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.504+0000 7fdfdd376640 1 -- 172.21.15.131:0/3531590329 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd814eb80 con 0x7fdfd8153250 2023-09-12T14:56:05.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.504+0000 7fdfd7fff640 1 --2- 172.21.15.131:0/3531590329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd81f13f0 0x7fdfd8152d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:05.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.504+0000 7fdfdd376640 1 --2- 172.21.15.131:0/3531590329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfd8153250 0x7fdfd81596c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdfd806c310 tx=0x7fdfd000b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:05.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.504+0000 7fdfd57fa640 1 -- 172.21.15.131:0/3531590329 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0015030 con 0x7fdfd8153250 2023-09-12T14:56:05.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.504+0000 7fdfd57fa640 1 -- 172.21.15.131:0/3531590329 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdfd0003980 con 0x7fdfd8153250 2023-09-12T14:56:05.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.504+0000 7fdfdedd9640 1 -- 172.21.15.131:0/3531590329 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd814ee60 con 0x7fdfd8153250 2023-09-12T14:56:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.504+0000 7fdfd57fa640 1 -- 172.21.15.131:0/3531590329 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0003d90 con 0x7fdfd8153250 2023-09-12T14:56:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.504+0000 7fdfdedd9640 1 -- 172.21.15.131:0/3531590329 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd81f1870 con 0x7fdfd8153250 2023-09-12T14:56:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.504+0000 7fdfdedd9640 1 -- 172.21.15.131:0/3531590329 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfd8073580 con 0x7fdfd8153250 2023-09-12T14:56:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.504+0000 7fdfd57fa640 1 -- 172.21.15.131:0/3531590329 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fdfd0018070 con 0x7fdfd8153250 2023-09-12T14:56:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.504+0000 7fdfd57fa640 1 --2- 172.21.15.131:0/3531590329 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdfb8065b50 0x7fdfb8068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.504+0000 7fdfd57fa640 1 -- 172.21.15.131:0/3531590329 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdfd0016070 con 0x7fdfd8153250 2023-09-12T14:56:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.504+0000 7fdfdcb75640 1 --2- 172.21.15.131:0/3531590329 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdfb8065b50 0x7fdfb8068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.504+0000 7fdfdcb75640 1 --2- 172.21.15.131:0/3531590329 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdfb8065b50 0x7fdfb8068010 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdfc00ae000 tx=0x7fdfc4005800 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:05.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.508+0000 7fdfd57fa640 1 -- 172.21.15.131:0/3531590329 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fdfd0050f50 con 0x7fdfd8153250 2023-09-12T14:56:05.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.712+0000 7fdfdedd9640 1 -- 172.21.15.131:0/3531590329 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fdfd80663b0 con 0x7fdfd8153250 2023-09-12T14:56:05.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.716+0000 7fdfd57fa640 1 -- 172.21.15.131:0/3531590329 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+147 (secure 0 0 0) 0x7fdfd0054570 con 0x7fdfd8153250 2023-09-12T14:56:05.720 INFO:teuthology.orchestra.run.smithi131.stdout:[client.0] 2023-09-12T14:56:05.721 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQCGewBlR6eoARAACciqMiYRB92w6zQMKfJGiA== 2023-09-12T14:56:05.721 INFO:teuthology.orchestra.run.smithi131.stdout: caps mds = "allow" 2023-09-12T14:56:05.721 INFO:teuthology.orchestra.run.smithi131.stdout: caps mon = "allow r" 2023-09-12T14:56:05.721 INFO:teuthology.orchestra.run.smithi131.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-09-12T14:56:05.721 INFO:teuthology.orchestra.run.smithi131.stderr:updated caps for client.0 2023-09-12T14:56:05.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.720+0000 7fdfdedd9640 1 -- 172.21.15.131:0/3531590329 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdfb8065b50 msgr2=0x7fdfb8068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:05.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.720+0000 7fdfdedd9640 1 --2- 172.21.15.131:0/3531590329 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdfb8065b50 0x7fdfb8068010 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdfc00ae000 tx=0x7fdfc4005800 comp rx=0 tx=0).stop 2023-09-12T14:56:05.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.720+0000 7fdfdedd9640 1 -- 172.21.15.131:0/3531590329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfd8153250 msgr2=0x7fdfd81596c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:05.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.720+0000 7fdfdedd9640 1 --2- 172.21.15.131:0/3531590329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfd8153250 0x7fdfd81596c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdfd806c310 tx=0x7fdfd000b620 comp rx=0 tx=0).stop 2023-09-12T14:56:05.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.720+0000 7fdfdedd9640 1 -- 172.21.15.131:0/3531590329 shutdown_connections 2023-09-12T14:56:05.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.720+0000 7fdfdedd9640 1 --2- 172.21.15.131:0/3531590329 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fdfb8065b50 0x7fdfb8068010 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:05.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.720+0000 7fdfdedd9640 1 --2- 172.21.15.131:0/3531590329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfd8153250 0x7fdfd81596c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:05.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.720+0000 7fdfdedd9640 1 --2- 172.21.15.131:0/3531590329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd81f13f0 0x7fdfd8152d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:05.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.720+0000 7fdfdedd9640 1 --2- 172.21.15.131:0/3531590329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfc40039b0 0x7fdfd81503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:05.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.720+0000 7fdfdedd9640 1 -- 172.21.15.131:0/3531590329 >> 172.21.15.131:0/3531590329 conn(0x7fdfd8069840 msgr2=0x7fdfd80cc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:05.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.720+0000 7fdfdedd9640 1 -- 172.21.15.131:0/3531590329 shutdown_connections 2023-09-12T14:56:05.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.720+0000 7fdfdedd9640 1 -- 172.21.15.131:0/3531590329 wait complete. 2023-09-12T14:56:05.738 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T14:56:05.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f862489e640 1 -- 172.21.15.131:0/4034075221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f862015a750 msgr2=0x7f862006f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:05.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f862489e640 1 --2- 172.21.15.131:0/4034075221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f862015a750 0x7f862006f6a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f862006dbb0 tx=0x7f861001c5b0 comp rx=0 tx=0).stop 2023-09-12T14:56:05.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f862489e640 1 -- 172.21.15.131:0/4034075221 shutdown_connections 2023-09-12T14:56:05.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f862489e640 1 --2- 172.21.15.131:0/4034075221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86200cc360 0x7f86200b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:05.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f862489e640 1 --2- 172.21.15.131:0/4034075221 >> v2:172.21.15.187:3300/0 conn(0x7f86200b00d0 0x7f862006fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:05.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f862489e640 1 --2- 172.21.15.131:0/4034075221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f862015a750 0x7f862006f6a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:05.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f862489e640 1 -- 172.21.15.131:0/4034075221 >> 172.21.15.131:0/4034075221 conn(0x7f862006b3a0 msgr2=0x7f862006b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:05.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f862489e640 1 -- 172.21.15.131:0/4034075221 shutdown_connections 2023-09-12T14:56:05.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f862489e640 1 -- 172.21.15.131:0/4034075221 wait complete. 2023-09-12T14:56:05.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f862489e640 1 Processor -- start 2023-09-12T14:56:05.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f862489e640 1 -- start start 2023-09-12T14:56:05.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f862489e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86200b00d0 0x7f8620141b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:05.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f862489e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86200cc360 0x7f8620157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:05.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f862489e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f862015a750 0x7f86200a76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:05.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f862489e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8620142050 con 0x7f86200cc360 2023-09-12T14:56:05.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f862489e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86201579b0 con 0x7f86200b00d0 2023-09-12T14:56:05.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f862489e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86200a6600 con 0x7f862015a750 2023-09-12T14:56:05.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f8617fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86200cc360 0x7f8620157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:05.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f8617fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86200cc360 0x7f8620157470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37490/0 (socket says 172.21.15.131:37490) 2023-09-12T14:56:05.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f8617fff640 1 -- 172.21.15.131:0/1741708726 learned_addr learned my addr 172.21.15.131:0/1741708726 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:05.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f861e59c640 1 --2- 172.21.15.131:0/1741708726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86200b00d0 0x7f8620141b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:05.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f8617fff640 1 -- 172.21.15.131:0/1741708726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f862015a750 msgr2=0x7f86200a76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:05.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f8617fff640 1 --2- 172.21.15.131:0/1741708726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f862015a750 0x7f86200a76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:05.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f8617fff640 1 -- 172.21.15.131:0/1741708726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86200b00d0 msgr2=0x7f8620141b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:05.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f8617fff640 1 --2- 172.21.15.131:0/1741708726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86200b00d0 0x7f8620141b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:05.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f8617fff640 1 -- 172.21.15.131:0/1741708726 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8610009560 con 0x7f86200cc360 2023-09-12T14:56:05.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f8617fff640 1 --2- 172.21.15.131:0/1741708726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86200cc360 0x7f8620157470 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f860800c970 tx=0x7f860800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:05.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f8616ffd640 1 -- 172.21.15.131:0/1741708726 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8608007bb0 con 0x7f86200cc360 2023-09-12T14:56:05.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f8616ffd640 1 -- 172.21.15.131:0/1741708726 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8608007e00 con 0x7f86200cc360 2023-09-12T14:56:05.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f862489e640 1 -- 172.21.15.131:0/1741708726 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86200a68e0 con 0x7f86200cc360 2023-09-12T14:56:05.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f8616ffd640 1 -- 172.21.15.131:0/1741708726 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8608005280 con 0x7f86200cc360 2023-09-12T14:56:05.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f862489e640 1 -- 172.21.15.131:0/1741708726 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86200a6ea0 con 0x7f86200cc360 2023-09-12T14:56:05.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.852+0000 7f862489e640 1 -- 172.21.15.131:0/1741708726 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8620065a90 con 0x7f86200cc360 2023-09-12T14:56:05.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.856+0000 7f8616ffd640 1 -- 172.21.15.131:0/1741708726 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f86080053e0 con 0x7f86200cc360 2023-09-12T14:56:05.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.856+0000 7f8616ffd640 1 --2- 172.21.15.131:0/1741708726 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8600065b20 0x7f8600067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:05.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.856+0000 7f8616ffd640 1 -- 172.21.15.131:0/1741708726 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86080838d0 con 0x7f86200cc360 2023-09-12T14:56:05.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.856+0000 7f861e59c640 1 --2- 172.21.15.131:0/1741708726 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8600065b20 0x7f8600067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:05.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.856+0000 7f861e59c640 1 --2- 172.21.15.131:0/1741708726 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8600065b20 0x7f8600067fe0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f86100ae000 tx=0x7f861001adf0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:05.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:05.860+0000 7f8616ffd640 1 -- 172.21.15.131:0/1741708726 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f8608054d40 con 0x7f86200cc360 2023-09-12T14:56:06.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.040+0000 7f862489e640 1 -- 172.21.15.131:0/1741708726 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f862006ab00 con 0x7f86200cc360 2023-09-12T14:56:06.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.040+0000 7f8616ffd640 1 -- 172.21.15.131:0/1741708726 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f860801d070 con 0x7f86200cc360 2023-09-12T14:56:06.045 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:56:06.045 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4676,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/3890238785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3890238785},{"type":"v1","addr":"172.21.15.187:6834","nonce":3890238785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4978,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1436257801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":1436257801},{"type":"v1","addr":"172.21.15.187:6835","nonce":1436257801}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4990,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/806466310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":806466310},{"type":"v1","addr":"172.21.15.131:6837","nonce":806466310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-12T14:55:56.870775+0000","modified":"2023-09-12T14:55:58.869201+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/871788191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":871788191},{"type":"v1","addr":"172.21.15.131:6835","nonce":871788191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-09-12T14:56:06.046 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 13 2023-09-12T14:56:06.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.044+0000 7f862489e640 1 -- 172.21.15.131:0/1741708726 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8600065b20 msgr2=0x7f8600067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:06.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.044+0000 7f862489e640 1 --2- 172.21.15.131:0/1741708726 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8600065b20 0x7f8600067fe0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f86100ae000 tx=0x7f861001adf0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.044+0000 7f862489e640 1 -- 172.21.15.131:0/1741708726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86200cc360 msgr2=0x7f8620157470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:06.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.044+0000 7f862489e640 1 --2- 172.21.15.131:0/1741708726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86200cc360 0x7f8620157470 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f860800c970 tx=0x7f860800ce30 comp rx=0 tx=0).stop 2023-09-12T14:56:06.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.044+0000 7f862489e640 1 -- 172.21.15.131:0/1741708726 shutdown_connections 2023-09-12T14:56:06.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.044+0000 7f862489e640 1 --2- 172.21.15.131:0/1741708726 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8600065b20 0x7f8600067fe0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.044+0000 7f862489e640 1 --2- 172.21.15.131:0/1741708726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f862015a750 0x7f86200a76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.044+0000 7f862489e640 1 --2- 172.21.15.131:0/1741708726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86200cc360 0x7f8620157470 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.044+0000 7f862489e640 1 --2- 172.21.15.131:0/1741708726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86200b00d0 0x7f8620141b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.044+0000 7f862489e640 1 -- 172.21.15.131:0/1741708726 >> 172.21.15.131:0/1741708726 conn(0x7f862006b3a0 msgr2=0x7f8620157c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:06.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.044+0000 7f862489e640 1 -- 172.21.15.131:0/1741708726 shutdown_connections 2023-09-12T14:56:06.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.044+0000 7f862489e640 1 -- 172.21.15.131:0/1741708726 wait complete. 2023-09-12T14:56:06.464 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-12T14:55:56.870775+0000', 'modified': '2023-09-12T14:55:58.869201+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5268}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5268': {'gid': 5268, 'name': 'a', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.131:6835/871788191', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 871788191}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 871788191}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-12T14:56:06.465 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-12T14:56:06.466 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-09-12T14:56:06.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.600+0000 7fa31ced9640 1 -- 172.21.15.131:0/136932174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3180b2600 msgr2=0x7fa3180c9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:06.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.600+0000 7fa31ced9640 1 --2- 172.21.15.131:0/136932174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3180b2600 0x7fa3180c9390 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa31806be30 tx=0x7fa30001c7b0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.600+0000 7fa31ced9640 1 -- 172.21.15.131:0/136932174 shutdown_connections 2023-09-12T14:56:06.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.600+0000 7fa31ced9640 1 --2- 172.21.15.131:0/136932174 >> v2:172.21.15.187:3300/0 conn(0x7fa3180c98d0 0x7fa3180af9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.600+0000 7fa31ced9640 1 --2- 172.21.15.131:0/136932174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3180b2600 0x7fa3180c9390 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.600+0000 7fa31ced9640 1 -- 172.21.15.131:0/136932174 >> 172.21.15.131:0/136932174 conn(0x7fa318069840 msgr2=0x7fa3180cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:06.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.600+0000 7fa31ced9640 1 -- 172.21.15.131:0/136932174 shutdown_connections 2023-09-12T14:56:06.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.600+0000 7fa31ced9640 1 -- 172.21.15.131:0/136932174 wait complete. 2023-09-12T14:56:06.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.600+0000 7fa31ced9640 1 Processor -- start 2023-09-12T14:56:06.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.600+0000 7fa31ced9640 1 -- start start 2023-09-12T14:56:06.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.600+0000 7fa31ced9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3180b2600 0x7fa318144f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:06.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.600+0000 7fa31ced9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3180c98d0 0x7fa3181474c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:06.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.600+0000 7fa31ced9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3181574d0 0x7fa3181446e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:06.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.600+0000 7fa31ced9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa318148820 con 0x7fa3181574d0 2023-09-12T14:56:06.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.600+0000 7fa31ced9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa318147ba0 con 0x7fa3180c98d0 2023-09-12T14:56:06.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.600+0000 7fa31ced9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa318147e80 con 0x7fa3180b2600 2023-09-12T14:56:06.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.600+0000 7fa316d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3181574d0 0x7fa3181446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:06.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.600+0000 7fa316d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3181574d0 0x7fa3181446e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37506/0 (socket says 172.21.15.131:37506) 2023-09-12T14:56:06.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.600+0000 7fa316d9d640 1 -- 172.21.15.131:0/1451934615 learned_addr learned my addr 172.21.15.131:0/1451934615 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:06.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.600+0000 7fa30dd9b640 1 --2- 172.21.15.131:0/1451934615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3180c98d0 0x7fa3181474c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:06.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.600+0000 7fa316d9d640 1 -- 172.21.15.131:0/1451934615 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3180b2600 msgr2=0x7fa318144f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:06.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.600+0000 7fa316d9d640 1 --2- 172.21.15.131:0/1451934615 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3180b2600 0x7fa318144f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.600+0000 7fa316d9d640 1 -- 172.21.15.131:0/1451934615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3180c98d0 msgr2=0x7fa3181474c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:06.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.600+0000 7fa316d9d640 1 --2- 172.21.15.131:0/1451934615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3180c98d0 0x7fa3181474c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.600+0000 7fa316d9d640 1 -- 172.21.15.131:0/1451934615 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3000094a0 con 0x7fa3181574d0 2023-09-12T14:56:06.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.600+0000 7fa30dd9b640 1 --2- 172.21.15.131:0/1451934615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3180c98d0 0x7fa3181474c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:06.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.604+0000 7fa316d9d640 1 --2- 172.21.15.131:0/1451934615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3181574d0 0x7fa3181446e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa31000ea20 tx=0x7fa31000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:06.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.604+0000 7fa30f7fe640 1 -- 172.21.15.131:0/1451934615 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31000cde0 con 0x7fa3181574d0 2023-09-12T14:56:06.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.604+0000 7fa30f7fe640 1 -- 172.21.15.131:0/1451934615 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa310004c40 con 0x7fa3181574d0 2023-09-12T14:56:06.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.604+0000 7fa31ced9640 1 -- 172.21.15.131:0/1451934615 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa318148160 con 0x7fa3181574d0 2023-09-12T14:56:06.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.604+0000 7fa30f7fe640 1 -- 172.21.15.131:0/1451934615 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa310005020 con 0x7fa3181574d0 2023-09-12T14:56:06.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.604+0000 7fa31ced9640 1 -- 172.21.15.131:0/1451934615 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa318144c20 con 0x7fa3181574d0 2023-09-12T14:56:06.618 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.604+0000 7fa30f7fe640 1 -- 172.21.15.131:0/1451934615 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fa310005180 con 0x7fa3181574d0 2023-09-12T14:56:06.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.604+0000 7fa30f7fe640 1 --2- 172.21.15.131:0/1451934615 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa2f8065b20 0x7fa2f8067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:06.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.604+0000 7fa30f7fe640 1 -- 172.21.15.131:0/1451934615 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa310083d10 con 0x7fa3181574d0 2023-09-12T14:56:06.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.604+0000 7fa31659c640 1 --2- 172.21.15.131:0/1451934615 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa2f8065b20 0x7fa2f8067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:06.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.604+0000 7fa31659c640 1 --2- 172.21.15.131:0/1451934615 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa2f8065b20 0x7fa2f8067fe0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa308007fd0 tx=0x7fa3080011a0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:06.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.604+0000 7fa320006000 1 Processor -- start 2023-09-12T14:56:06.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.604+0000 7fa320006000 1 -- start start 2023-09-12T14:56:06.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa320006000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559013bafed0 0x559013bb02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:06.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa320006000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559013bb07f0 0x559013b65df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:06.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa320006000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013b66330 0x559013b66710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:06.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa320006000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590139bd510 con 0x559013b66330 2023-09-12T14:56:06.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa320006000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590139baa40 con 0x559013bafed0 2023-09-12T14:56:06.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa320006000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590139b6330 con 0x559013bb07f0 2023-09-12T14:56:06.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa316d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013b66330 0x559013b66710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:06.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa316d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013b66330 0x559013b66710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37512/0 (socket says 172.21.15.131:37512) 2023-09-12T14:56:06.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa316d9d640 1 -- 172.21.15.131:0/382592831 learned_addr learned my addr 172.21.15.131:0/382592831 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:06.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa31659c640 1 --2- 172.21.15.131:0/382592831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559013bafed0 0x559013bb02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:06.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa30dd9b640 1 --2- 172.21.15.131:0/382592831 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559013bb07f0 0x559013b65df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:06.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa316d9d640 1 -- 172.21.15.131:0/382592831 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559013bb07f0 msgr2=0x559013b65df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T14:56:06.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa316d9d640 1 --2- 172.21.15.131:0/382592831 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559013bb07f0 0x559013b65df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa316d9d640 1 -- 172.21.15.131:0/382592831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559013bafed0 msgr2=0x559013bb02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T14:56:06.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa316d9d640 1 --2- 172.21.15.131:0/382592831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559013bafed0 0x559013bb02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa316d9d640 1 -- 172.21.15.131:0/382592831 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590136b0890 con 0x559013b66330 2023-09-12T14:56:06.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa31659c640 1 --2- 172.21.15.131:0/382592831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559013bafed0 0x559013bb02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:56:06.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa316d9d640 1 --2- 172.21.15.131:0/382592831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013b66330 0x559013b66710 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa3100822f0 tx=0x7fa310055b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57854bf38266ef42 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:06.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa2f7fff640 1 -- 172.21.15.131:0/382592831 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31005d390 con 0x559013b66330 2023-09-12T14:56:06.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa2f7fff640 1 -- 172.21.15.131:0/382592831 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa310053750 con 0x559013b66330 2023-09-12T14:56:06.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa2f7fff640 1 -- 172.21.15.131:0/382592831 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31005b0c0 con 0x559013b66330 2023-09-12T14:56:06.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa320006000 1 -- 172.21.15.131:0/382592831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013b66330 msgr2=0x559013b66710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:06.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa320006000 1 --2- 172.21.15.131:0/382592831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013b66330 0x559013b66710 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa3100822f0 tx=0x7fa310055b70 comp rx=0 tx=0).stop 2023-09-12T14:56:06.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa320006000 1 -- 172.21.15.131:0/382592831 shutdown_connections 2023-09-12T14:56:06.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa320006000 1 --2- 172.21.15.131:0/382592831 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559013bb07f0 0x559013b65df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa320006000 1 --2- 172.21.15.131:0/382592831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013b66330 0x559013b66710 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa320006000 1 --2- 172.21.15.131:0/382592831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559013bafed0 0x559013bb02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa320006000 1 -- 172.21.15.131:0/382592831 >> 172.21.15.131:0/382592831 conn(0x559013b6c490 msgr2=0x559013b646d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:06.626 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa320006000 1 -- 172.21.15.131:0/382592831 shutdown_connections 2023-09-12T14:56:06.626 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa320006000 1 -- 172.21.15.131:0/382592831 wait complete. 2023-09-12T14:56:06.626 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa320006000 1 -- start start 2023-09-12T14:56:06.626 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa320006000 1 Processor -- start 2023-09-12T14:56:06.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa320006000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559013b66330 0x559013b70e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:06.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa320006000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559013bafed0 0x559013b713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:06.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa320006000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013bb07f0 0x559013b71910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:06.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa320006000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559013970ed0 con 0x559013bb07f0 2023-09-12T14:56:06.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa320006000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x559013970b70 con 0x559013bafed0 2023-09-12T14:56:06.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa320006000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x559013671940 con 0x559013b66330 2023-09-12T14:56:06.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa320006000 10 client.0 _ll_register_callbacks cb 0x559013a844a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T14:56:06.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa316d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013bb07f0 0x559013b71910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:06.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa316d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013bb07f0 0x559013b71910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37522/0 (socket says 172.21.15.131:37522) 2023-09-12T14:56:06.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa316d9d640 1 -- 172.21.15.131:0/3071680278 learned_addr learned my addr 172.21.15.131:0/3071680278 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:06.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa31659c640 1 --2- 172.21.15.131:0/3071680278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559013b66330 0x559013b70e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:06.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa30dd9b640 1 --2- 172.21.15.131:0/3071680278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559013bafed0 0x559013b713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:06.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa316d9d640 1 -- 172.21.15.131:0/3071680278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559013b66330 msgr2=0x559013b70e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T14:56:06.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa316d9d640 1 --2- 172.21.15.131:0/3071680278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559013b66330 0x559013b70e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa316d9d640 1 -- 172.21.15.131:0/3071680278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559013bafed0 msgr2=0x559013b713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T14:56:06.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa316d9d640 1 --2- 172.21.15.131:0/3071680278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559013bafed0 0x559013b713d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa316d9d640 1 -- 172.21.15.131:0/3071680278 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa318144c20 con 0x559013bb07f0 2023-09-12T14:56:06.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa316d9d640 1 --2- 172.21.15.131:0/3071680278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013bb07f0 0x559013b71910 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa3100518b0 tx=0x7fa3100516f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf2a96c2ad7ee9f server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:06.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa2f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-12T14:56:06.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa2f57fa640 1 -- 172.21.15.131:0/3071680278 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31005d390 con 0x559013bb07f0 2023-09-12T14:56:06.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa320006000 1 -- 172.21.15.131:0/3071680278 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55901396d880 con 0x559013bb07f0 2023-09-12T14:56:06.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa2f57fa640 1 -- 172.21.15.131:0/3071680278 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa31005b050 con 0x559013bb07f0 2023-09-12T14:56:06.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa2f57fa640 1 -- 172.21.15.131:0/3071680278 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa31005b210 con 0x559013bb07f0 2023-09-12T14:56:06.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa320006000 10 client.5478 fetch_fsmap learned FSMap version 13 2023-09-12T14:56:06.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.608+0000 7fa320006000 1 -- 172.21.15.131:0/3071680278 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x559013979290 con 0x559013bb07f0 2023-09-12T14:56:06.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.612+0000 7fa2f57fa640 1 -- 172.21.15.131:0/3071680278 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x55901396d880 con 0x559013bb07f0 2023-09-12T14:56:06.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.612+0000 7fa2f57fa640 1 -- 172.21.15.131:0/3071680278 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 13) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa310018030 con 0x559013bb07f0 2023-09-12T14:56:06.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.612+0000 7fa320006000 10 client.5478 fetch_fsmap finished waiting for FSMap version 13 2023-09-12T14:56:06.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.612+0000 7fa320006000 10 client.5478 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-12T14:56:06.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.612+0000 7fa320006000 20 client.5478 populate_metadata read hostname 'smithi131' 2023-09-12T14:56:06.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.612+0000 7fa320006000 1 --2- 172.21.15.131:0/3071680278 >> [v2:172.21.15.131:6834/871788191,v1:172.21.15.131:6835/871788191] conn(0x559013bbf170 0x559013bc1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:06.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.612+0000 7fa320006000 4 client.5478 mds_command: new command op to 5268 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-12T14:56:06.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.612+0000 7fa320006000 1 -- 172.21.15.131:0/3071680278 --> [v2:172.21.15.131:6834/871788191,v1:172.21.15.131:6835/871788191] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559013976700 con 0x559013bbf170 2023-09-12T14:56:06.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.612+0000 7fa31659c640 1 --2- 172.21.15.131:0/3071680278 >> [v2:172.21.15.131:6834/871788191,v1:172.21.15.131:6835/871788191] conn(0x559013bbf170 0x559013bc1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:06.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.612+0000 7fa31659c640 1 --2- 172.21.15.131:0/3071680278 >> [v2:172.21.15.131:6834/871788191,v1:172.21.15.131:6835/871788191] conn(0x559013bbf170 0x559013bc1560 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b8c820b3d2a8c45 server_cookie=f57cde26dbc9fa38 in_seq=0 out_seq=0 2023-09-12T14:56:06.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.612+0000 7fa2f57fa640 10 client.5478 ms_handle_connect on v2:172.21.15.131:6834/871788191 2023-09-12T14:56:06.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.612+0000 7fa2f57fa640 1 -- 172.21.15.131:0/3071680278 <== mds.0 v2:172.21.15.131:6834/871788191 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559013976700 con 0x559013bbf170 2023-09-12T14:56:06.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.612+0000 7fa2f57fa640 10 client.5478 handle_command_reply: tid=0 2023-09-12T14:56:06.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.612+0000 7fa320006000 1 client.5478 shutdown 2023-09-12T14:56:06.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.612+0000 7fa320006000 1 -- 172.21.15.131:0/3071680278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013bb07f0 msgr2=0x559013b71910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:06.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.612+0000 7fa320006000 1 --2- 172.21.15.131:0/3071680278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013bb07f0 0x559013b71910 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa3100518b0 tx=0x7fa3100516f0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.612+0000 7fa320006000 1 -- 172.21.15.131:0/3071680278 shutdown_connections 2023-09-12T14:56:06.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.612+0000 7fa320006000 1 --2- 172.21.15.131:0/3071680278 >> [v2:172.21.15.131:6834/871788191,v1:172.21.15.131:6835/871788191] conn(0x559013bbf170 0x559013bc1560 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.612+0000 7fa320006000 1 --2- 172.21.15.131:0/3071680278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013bb07f0 0x559013b71910 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.612+0000 7fa2f57fa640 0 client.5478 ms_handle_reset on v2:172.21.15.131:6834/871788191 2023-09-12T14:56:06.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.612+0000 7fa320006000 1 --2- 172.21.15.131:0/3071680278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559013bafed0 0x559013b713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.612+0000 7fa320006000 1 --2- 172.21.15.131:0/3071680278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559013b66330 0x559013b70e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.612+0000 7fa320006000 1 -- 172.21.15.131:0/3071680278 >> 172.21.15.131:0/3071680278 conn(0x559013b6c490 msgr2=0x559013b64dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:06.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.612+0000 7fa320006000 1 -- 172.21.15.131:0/3071680278 shutdown_connections 2023-09-12T14:56:06.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.612+0000 7fa320006000 1 -- 172.21.15.131:0/3071680278 wait complete. 2023-09-12T14:56:06.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.612+0000 7fa320006000 20 client.5478 trim_cache size 0 max 16384 2023-09-12T14:56:06.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa320006000 1 Processor -- start 2023-09-12T14:56:06.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa320006000 1 -- start start 2023-09-12T14:56:06.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa320006000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013b66330 0x559013c0a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:06.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa320006000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559013bafed0 0x559013c0aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:06.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa320006000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559013bb07f0 0x559013c0b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:06.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa320006000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559013951db0 con 0x559013b66330 2023-09-12T14:56:06.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa320006000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590139af6e0 con 0x559013bb07f0 2023-09-12T14:56:06.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa320006000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590139b51c0 con 0x559013bafed0 2023-09-12T14:56:06.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa31659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013b66330 0x559013c0a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:06.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa31659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013b66330 0x559013c0a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37538/0 (socket says 172.21.15.131:37538) 2023-09-12T14:56:06.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa31659c640 1 -- 172.21.15.131:0/1775240322 learned_addr learned my addr 172.21.15.131:0/1775240322 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:06.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa31659c640 1 -- 172.21.15.131:0/1775240322 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559013bafed0 msgr2=0x559013c0aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T14:56:06.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa30dd9b640 1 --2- 172.21.15.131:0/1775240322 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559013bafed0 0x559013c0aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:06.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa316d9d640 1 --2- 172.21.15.131:0/1775240322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559013bb07f0 0x559013c0b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:06.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa31659c640 1 --2- 172.21.15.131:0/1775240322 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559013bafed0 0x559013c0aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa31659c640 1 -- 172.21.15.131:0/1775240322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559013bb07f0 msgr2=0x559013c0b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T14:56:06.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa31659c640 1 --2- 172.21.15.131:0/1775240322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559013bb07f0 0x559013c0b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa31659c640 1 -- 172.21.15.131:0/1775240322 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55901395f500 con 0x559013b66330 2023-09-12T14:56:06.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa316d9d640 1 --2- 172.21.15.131:0/1775240322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559013bb07f0 0x559013c0b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:06.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa31659c640 1 --2- 172.21.15.131:0/1775240322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013b66330 0x559013c0a970 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa308001380 tx=0x7fa308012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755d744ebf89cc78 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:06.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa30cd99640 1 -- 172.21.15.131:0/1775240322 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3080252e0 con 0x559013b66330 2023-09-12T14:56:06.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa30cd99640 1 -- 172.21.15.131:0/1775240322 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa30801c040 con 0x559013b66330 2023-09-12T14:56:06.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa30cd99640 1 -- 172.21.15.131:0/1775240322 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa308014e50 con 0x559013b66330 2023-09-12T14:56:06.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa320006000 1 -- 172.21.15.131:0/1775240322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013b66330 msgr2=0x559013c0a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:06.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa320006000 1 --2- 172.21.15.131:0/1775240322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013b66330 0x559013c0a970 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa308001380 tx=0x7fa308012000 comp rx=0 tx=0).stop 2023-09-12T14:56:06.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa320006000 1 -- 172.21.15.131:0/1775240322 shutdown_connections 2023-09-12T14:56:06.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa320006000 1 --2- 172.21.15.131:0/1775240322 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559013bafed0 0x559013c0aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa320006000 1 --2- 172.21.15.131:0/1775240322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559013bb07f0 0x559013c0b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa320006000 1 --2- 172.21.15.131:0/1775240322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013b66330 0x559013c0a970 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa320006000 1 -- 172.21.15.131:0/1775240322 >> 172.21.15.131:0/1775240322 conn(0x559013b6c490 msgr2=0x559013bc2470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:06.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa320006000 1 -- 172.21.15.131:0/1775240322 shutdown_connections 2023-09-12T14:56:06.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa320006000 1 -- 172.21.15.131:0/1775240322 wait complete. 2023-09-12T14:56:06.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa320006000 1 -- start start 2023-09-12T14:56:06.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa320006000 1 Processor -- start 2023-09-12T14:56:06.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa320006000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559013b66330 0x559013bae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:06.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa320006000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013bafed0 0x559013c0ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:06.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa320006000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559013bb07f0 0x559013c0b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:06.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa320006000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559013b710c0 con 0x559013bafed0 2023-09-12T14:56:06.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa320006000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x559013970ed0 con 0x559013bb07f0 2023-09-12T14:56:06.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.628+0000 7fa320006000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x559013671940 con 0x559013b66330 2023-09-12T14:56:06.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa320006000 10 client.0 _ll_register_callbacks cb 0x559013955e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-12T14:56:06.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa30dd9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013bafed0 0x559013c0ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:06.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa30dd9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013bafed0 0x559013c0ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37544/0 (socket says 172.21.15.131:37544) 2023-09-12T14:56:06.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa30dd9b640 1 -- 172.21.15.131:0/4128260966 learned_addr learned my addr 172.21.15.131:0/4128260966 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:06.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa316d9d640 1 --2- 172.21.15.131:0/4128260966 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559013bb07f0 0x559013c0b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:06.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa31659c640 1 --2- 172.21.15.131:0/4128260966 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559013b66330 0x559013bae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:06.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa30dd9b640 1 -- 172.21.15.131:0/4128260966 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559013b66330 msgr2=0x559013bae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T14:56:06.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa30dd9b640 1 --2- 172.21.15.131:0/4128260966 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559013b66330 0x559013bae9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa30dd9b640 1 -- 172.21.15.131:0/4128260966 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559013bb07f0 msgr2=0x559013c0b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-12T14:56:06.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa30dd9b640 1 --2- 172.21.15.131:0/4128260966 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559013bb07f0 0x559013c0b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa30dd9b640 1 -- 172.21.15.131:0/4128260966 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa304003530 con 0x559013bafed0 2023-09-12T14:56:06.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa316d9d640 1 --2- 172.21.15.131:0/4128260966 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559013bb07f0 0x559013c0b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:06.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa2f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-09-12T14:56:06.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa30dd9b640 1 --2- 172.21.15.131:0/4128260966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013bafed0 0x559013c0ab80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa3000193c0 tx=0x7fa3000a56c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167a27008144e05a server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:06.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa2f57fa640 1 -- 172.21.15.131:0/4128260966 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3000a5dd0 con 0x559013bafed0 2023-09-12T14:56:06.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa2f57fa640 1 -- 172.21.15.131:0/4128260966 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa300004040 con 0x559013bafed0 2023-09-12T14:56:06.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa320006000 1 -- 172.21.15.131:0/4128260966 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559013b69aa0 con 0x559013bafed0 2023-09-12T14:56:06.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa2f57fa640 1 -- 172.21.15.131:0/4128260966 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3000a58f0 con 0x559013bafed0 2023-09-12T14:56:06.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa320006000 10 client.5484 fetch_fsmap learned FSMap version 13 2023-09-12T14:56:06.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa320006000 1 -- 172.21.15.131:0/4128260966 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x5590136b0cc0 con 0x559013bafed0 2023-09-12T14:56:06.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa2f57fa640 1 -- 172.21.15.131:0/4128260966 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x559013b69aa0 con 0x559013bafed0 2023-09-12T14:56:06.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa2f57fa640 1 -- 172.21.15.131:0/4128260966 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 13) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa300004310 con 0x559013bafed0 2023-09-12T14:56:06.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa320006000 10 client.5484 fetch_fsmap finished waiting for FSMap version 13 2023-09-12T14:56:06.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa320006000 10 client.5484 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-12T14:56:06.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa320006000 20 client.5484 populate_metadata read hostname 'smithi131' 2023-09-12T14:56:06.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa320006000 1 --2- 172.21.15.131:0/4128260966 >> [v2:172.21.15.131:6834/871788191,v1:172.21.15.131:6835/871788191] conn(0x559013c0ee10 0x559013c0f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:06.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa320006000 4 client.5484 mds_command: new command op to 5268 tid=0[{"prefix": "status"}] 2023-09-12T14:56:06.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa320006000 1 -- 172.21.15.131:0/4128260966 --> [v2:172.21.15.131:6834/871788191,v1:172.21.15.131:6835/871788191] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5590139bc310 con 0x559013c0ee10 2023-09-12T14:56:06.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa31659c640 1 --2- 172.21.15.131:0/4128260966 >> [v2:172.21.15.131:6834/871788191,v1:172.21.15.131:6835/871788191] conn(0x559013c0ee10 0x559013c0f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:06.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa31659c640 1 --2- 172.21.15.131:0/4128260966 >> [v2:172.21.15.131:6834/871788191,v1:172.21.15.131:6835/871788191] conn(0x559013c0ee10 0x559013c0f1f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=554f5c69453c111a server_cookie=376ba45f9a3b1e40 in_seq=0 out_seq=0 2023-09-12T14:56:06.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa2f57fa640 10 client.5484 ms_handle_connect on v2:172.21.15.131:6834/871788191 2023-09-12T14:56:06.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa2f57fa640 1 -- 172.21.15.131:0/4128260966 <== mds.0 v2:172.21.15.131:6834/871788191 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5590139bc310 con 0x559013c0ee10 2023-09-12T14:56:06.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa2f57fa640 10 client.5484 handle_command_reply: tid=0 2023-09-12T14:56:06.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa320006000 1 client.5484 shutdown 2023-09-12T14:56:06.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa320006000 1 -- 172.21.15.131:0/4128260966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013bafed0 msgr2=0x559013c0ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:06.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa320006000 1 --2- 172.21.15.131:0/4128260966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013bafed0 0x559013c0ab80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa3000193c0 tx=0x7fa3000a56c0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa320006000 1 -- 172.21.15.131:0/4128260966 shutdown_connections 2023-09-12T14:56:06.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa320006000 1 --2- 172.21.15.131:0/4128260966 >> [v2:172.21.15.131:6834/871788191,v1:172.21.15.131:6835/871788191] conn(0x559013c0ee10 0x559013c0f1f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa320006000 1 --2- 172.21.15.131:0/4128260966 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559013bb07f0 0x559013c0b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa320006000 1 --2- 172.21.15.131:0/4128260966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559013bafed0 0x559013c0ab80 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa2f57fa640 0 client.5484 ms_handle_reset on v2:172.21.15.131:6834/871788191 2023-09-12T14:56:06.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa320006000 1 --2- 172.21.15.131:0/4128260966 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559013b66330 0x559013bae9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa320006000 1 -- 172.21.15.131:0/4128260966 >> 172.21.15.131:0/4128260966 conn(0x559013b6c490 msgr2=0x559013bc2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:06.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa320006000 1 -- 172.21.15.131:0/4128260966 shutdown_connections 2023-09-12T14:56:06.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa320006000 1 -- 172.21.15.131:0/4128260966 wait complete. 2023-09-12T14:56:06.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa320006000 20 client.5484 trim_cache size 0 max 16384 2023-09-12T14:56:06.662 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-09-12T14:56:06.662 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "6efb91e0-47b4-40bb-ae12-7c7ea0e893df", 2023-09-12T14:56:06.662 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2023-09-12T14:56:06.662 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 5268, 2023-09-12T14:56:06.663 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2023-09-12T14:56:06.663 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2023-09-12T14:56:06.663 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2023-09-12T14:56:06.663 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 9.1264538280000007, 2023-09-12T14:56:06.663 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 13, 2023-09-12T14:56:06.664 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 33, 2023-09-12T14:56:06.664 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 33, 2023-09-12T14:56:06.664 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 16.271076828999998 2023-09-12T14:56:06.664 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-09-12T14:56:06.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa31ced9640 1 -- 172.21.15.131:0/1451934615 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa2f8065b20 msgr2=0x7fa2f8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:06.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa31ced9640 1 --2- 172.21.15.131:0/1451934615 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa2f8065b20 0x7fa2f8067fe0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa308007fd0 tx=0x7fa3080011a0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa31ced9640 1 -- 172.21.15.131:0/1451934615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3181574d0 msgr2=0x7fa3181446e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:06.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa31ced9640 1 --2- 172.21.15.131:0/1451934615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3181574d0 0x7fa3181446e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa31000ea20 tx=0x7fa31000eee0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa31ced9640 1 -- 172.21.15.131:0/1451934615 shutdown_connections 2023-09-12T14:56:06.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa31ced9640 1 --2- 172.21.15.131:0/1451934615 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa2f8065b20 0x7fa2f8067fe0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa31ced9640 1 --2- 172.21.15.131:0/1451934615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3181574d0 0x7fa3181446e0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa31ced9640 1 --2- 172.21.15.131:0/1451934615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3180c98d0 0x7fa3181474c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa31ced9640 1 --2- 172.21.15.131:0/1451934615 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3180b2600 0x7fa318144f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa31ced9640 1 -- 172.21.15.131:0/1451934615 >> 172.21.15.131:0/1451934615 conn(0x7fa318069840 msgr2=0x7fa31806b430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:06.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa31ced9640 1 -- 172.21.15.131:0/1451934615 shutdown_connections 2023-09-12T14:56:06.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.632+0000 7fa31ced9640 1 -- 172.21.15.131:0/1451934615 wait complete. 2023-09-12T14:56:06.668 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-12T14:56:06.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.776+0000 7f172bdf4640 1 -- 172.21.15.131:0/2230564094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17241574d0 msgr2=0x7f17240cacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:06.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.776+0000 7f172bdf4640 1 --2- 172.21.15.131:0/2230564094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17241574d0 0x7f17240cacd0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1720009bc0 tx=0x7f172001cb10 comp rx=0 tx=0).stop 2023-09-12T14:56:06.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.776+0000 7f172bdf4640 1 -- 172.21.15.131:0/2230564094 shutdown_connections 2023-09-12T14:56:06.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.776+0000 7f172bdf4640 1 --2- 172.21.15.131:0/2230564094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17241574d0 0x7f17240cacd0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.776+0000 7f172bdf4640 1 -- 172.21.15.131:0/2230564094 >> 172.21.15.131:0/2230564094 conn(0x7f1724069840 msgr2=0x7f17240cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:06.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.776+0000 7f172bdf4640 1 -- 172.21.15.131:0/2230564094 shutdown_connections 2023-09-12T14:56:06.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.776+0000 7f172bdf4640 1 -- 172.21.15.131:0/2230564094 wait complete. 2023-09-12T14:56:06.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.780+0000 7f172bdf4640 1 Processor -- start 2023-09-12T14:56:06.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.780+0000 7f172bdf4640 1 -- start start 2023-09-12T14:56:06.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.780+0000 7f172bdf4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17240b2600 0x7f17241595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:06.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.780+0000 7f172bdf4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17240c9390 0x7f1724144170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:06.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.780+0000 7f172bdf4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17241574d0 0x7f172414b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:06.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.780+0000 7f172bdf4640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17241536a0 con 0x7f17240c9390 2023-09-12T14:56:06.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.780+0000 7f172bdf4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1724152c70 con 0x7f17240b2600 2023-09-12T14:56:06.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.780+0000 7f172bdf4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1724158ae0 con 0x7f17241574d0 2023-09-12T14:56:06.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.780+0000 7f172938f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17240c9390 0x7f1724144170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:06.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.780+0000 7f172938f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17240c9390 0x7f1724144170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37568/0 (socket says 172.21.15.131:37568) 2023-09-12T14:56:06.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.780+0000 7f172938f640 1 -- 172.21.15.131:0/991654821 learned_addr learned my addr 172.21.15.131:0/991654821 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:06.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.780+0000 7f1729b90640 1 --2- 172.21.15.131:0/991654821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17240b2600 0x7f17241595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:06.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.780+0000 7f172a391640 1 --2- 172.21.15.131:0/991654821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17241574d0 0x7f172414b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:06.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.780+0000 7f172938f640 1 -- 172.21.15.131:0/991654821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17241574d0 msgr2=0x7f172414b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:06.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.780+0000 7f172938f640 1 --2- 172.21.15.131:0/991654821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17241574d0 0x7f172414b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.780+0000 7f172938f640 1 -- 172.21.15.131:0/991654821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17240b2600 msgr2=0x7f17241595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:06.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.780+0000 7f172938f640 1 --2- 172.21.15.131:0/991654821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17240b2600 0x7f17241595c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.780+0000 7f172938f640 1 -- 172.21.15.131:0/991654821 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1720009820 con 0x7f17240c9390 2023-09-12T14:56:06.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.780+0000 7f172938f640 1 --2- 172.21.15.131:0/991654821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17240c9390 0x7f1724144170 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f171400c480 tx=0x7f171400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:06.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.780+0000 7f171a7fc640 1 -- 172.21.15.131:0/991654821 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714005480 con 0x7f17240c9390 2023-09-12T14:56:06.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.780+0000 7f171a7fc640 1 -- 172.21.15.131:0/991654821 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1714003600 con 0x7f17240c9390 2023-09-12T14:56:06.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.780+0000 7f171a7fc640 1 -- 172.21.15.131:0/991654821 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714003a10 con 0x7f17240c9390 2023-09-12T14:56:06.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.780+0000 7f172bdf4640 1 -- 172.21.15.131:0/991654821 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1724158dc0 con 0x7f17240c9390 2023-09-12T14:56:06.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.780+0000 7f172bdf4640 1 -- 172.21.15.131:0/991654821 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1724152f50 con 0x7f17240c9390 2023-09-12T14:56:06.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.780+0000 7f172bdf4640 1 -- 172.21.15.131:0/991654821 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17241592e0 con 0x7f17240c9390 2023-09-12T14:56:06.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.780+0000 7f171a7fc640 1 -- 172.21.15.131:0/991654821 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f1714003b70 con 0x7f17240c9390 2023-09-12T14:56:06.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.784+0000 7f171a7fc640 1 --2- 172.21.15.131:0/991654821 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1708065b20 0x7f1708067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:06.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.784+0000 7f171a7fc640 1 -- 172.21.15.131:0/991654821 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1714082ff0 con 0x7f17240c9390 2023-09-12T14:56:06.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.784+0000 7f1729b90640 1 --2- 172.21.15.131:0/991654821 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1708065b20 0x7f1708067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:06.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.784+0000 7f1729b90640 1 --2- 172.21.15.131:0/991654821 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1708065b20 0x7f1708067fe0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f17200af000 tx=0x7f172001b350 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:06.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.788+0000 7f171a7fc640 1 -- 172.21.15.131:0/991654821 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f17241592e0 con 0x7f17240c9390 2023-09-12T14:56:06.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.960+0000 7f172bdf4640 1 -- 172.21.15.131:0/991654821 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f17241592e0 con 0x7f17240c9390 2023-09-12T14:56:06.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.960+0000 7f171a7fc640 1 -- 172.21.15.131:0/991654821 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+73 (secure 0 0 0) 0x7f17241592e0 con 0x7f17240c9390 2023-09-12T14:56:06.965 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-12T14:56:06.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.964+0000 7f172bdf4640 1 -- 172.21.15.131:0/991654821 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1708065b20 msgr2=0x7f1708067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:06.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.964+0000 7f172bdf4640 1 --2- 172.21.15.131:0/991654821 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1708065b20 0x7f1708067fe0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f17200af000 tx=0x7f172001b350 comp rx=0 tx=0).stop 2023-09-12T14:56:06.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.964+0000 7f172bdf4640 1 -- 172.21.15.131:0/991654821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17240c9390 msgr2=0x7f1724144170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:06.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.964+0000 7f172bdf4640 1 --2- 172.21.15.131:0/991654821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17240c9390 0x7f1724144170 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f171400c480 tx=0x7f171400c940 comp rx=0 tx=0).stop 2023-09-12T14:56:06.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.964+0000 7f172bdf4640 1 -- 172.21.15.131:0/991654821 shutdown_connections 2023-09-12T14:56:06.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.964+0000 7f172bdf4640 1 --2- 172.21.15.131:0/991654821 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1708065b20 0x7f1708067fe0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.964+0000 7f172bdf4640 1 --2- 172.21.15.131:0/991654821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17241574d0 0x7f172414b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.964+0000 7f172bdf4640 1 --2- 172.21.15.131:0/991654821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17240c9390 0x7f1724144170 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.964+0000 7f172bdf4640 1 --2- 172.21.15.131:0/991654821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17240b2600 0x7f17241595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:06.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.964+0000 7f172bdf4640 1 -- 172.21.15.131:0/991654821 >> 172.21.15.131:0/991654821 conn(0x7f1724069840 msgr2=0x7f17240ae690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:06.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.964+0000 7f172bdf4640 1 -- 172.21.15.131:0/991654821 shutdown_connections 2023-09-12T14:56:06.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:06.964+0000 7f172bdf4640 1 -- 172.21.15.131:0/991654821 wait complete. 2023-09-12T14:56:06.982 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-12T14:56:06.982 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-12T14:56:06.983 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2023-09-12T14:56:06.983 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-12T14:56:06.983 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-12T14:56:06.984 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-12T14:56:06.984 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-12T14:56:06.984 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-12T14:56:06.985 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-12T14:56:06.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:06.986 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2023-09-12T14:56:06.991 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-12T14:56:06.991 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-12T14:56:06.991 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2023-09-12T14:56:06.992 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:56:06.992 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2023-09-12T14:56:06.992 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:56:06.992 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-12T14:56:06.992 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:56:06.993 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2023-09-12T14:56:06.993 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-12T14:56:06.993 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:56:06.993 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2023-09-12T14:56:06.993 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-12T14:56:06.994 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:56:06.994 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-12T14:56:06.994 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:56:06.994 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-09-12T14:56:06.994 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42429sec preferred_lft 42429sec 2023-09-12T14:56:06.995 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2023-09-12T14:56:06.995 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:56:06.995 INFO:teuthology.orchestra.run.smithi131.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-09-12T14:56:06.995 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 02:42:21:fb:7e:42 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:56:06.995 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-09-12T14:56:06.996 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:56:06.996 INFO:teuthology.orchestra.run.smithi131.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-12T14:56:06.996 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-09-12T14:56:06.996 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-12T14:56:06.996 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:56:06.997 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-09-12T14:56:06.997 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:56:06.997 INFO:teuthology.orchestra.run.smithi131.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-12T14:56:06.997 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-12T14:56:06.997 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-09-12T14:56:06.998 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:56:06.998 INFO:teuthology.orchestra.run.smithi131.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-12T14:56:06.998 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-12T14:56:06.998 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-09-12T14:56:06.998 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:56:06.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:06.999 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-12T14:56:07.045 INFO:teuthology.orchestra.run.smithi131.stdout:1 2023-09-12T14:56:07.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:07.047 DEBUG:teuthology.orchestra.run.smithi131:> route 2023-09-12T14:56:07.096 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2023-09-12T14:56:07.097 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-12T14:56:07.097 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-09-12T14:56:07.097 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-12T14:56:07.097 INFO:teuthology.orchestra.run.smithi131.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-09-12T14:56:07.098 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-12T14:56:07.098 INFO:teuthology.orchestra.run.smithi131.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-12T14:56:07.098 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-12T14:56:07.098 INFO:teuthology.orchestra.run.smithi131.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-12T14:56:07.099 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-12T14:56:07.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:07.099 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-09-12T14:56:07.100 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-09-12T14:56:07.100 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-12T14:56:07.100 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-12T14:56:07.101 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-12T14:56:07.101 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-09-12T14:56:07.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:07.170 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2023-09-12T14:56:07.218 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-09-12T14:56:07.218 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-12T14:56:07.219 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-12T14:56:07.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:07.219 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2023-09-12T14:56:07.274 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-12T14:56:07.275 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-12T14:56:07.275 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-12T14:56:07.322 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-12T14:56:07.323 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-12T14:56:07.323 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-12T14:56:07.370 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2023-09-12T14:56:07.431 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-12T14:56:07.432 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-12T14:56:07.489 INFO:teuthology.orchestra.run.smithi131.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-12T14:56:07.490 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-12T14:56:07.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:07.492 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2023-09-12T14:56:07.542 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-12T14:56:07.542 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-12T14:56:07.591 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2023-09-12T14:56:07.605 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-12T14:56:07.606 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-12T14:56:07.662 INFO:teuthology.orchestra.run.smithi131.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-12T14:56:07.663 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-12T14:56:07.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:07.664 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2023-09-12T14:56:07.670 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:2023-09-12T14:56:07.668+0000 7f413e151580 -1 init, newargv = 0x56172370c4f0 newargc=15 2023-09-12T14:56:07.671 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[49189]: starting ceph client 2023-09-12T14:56:07.675 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[49189]: starting fuse 2023-09-12T14:56:08.777 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2023-09-12T14:56:08.794 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-12T14:56:08.795 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-12T14:56:08.849 INFO:teuthology.orchestra.run.smithi131.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-12T14:56:08.851 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-12T14:56:08.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:08.852 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2023-09-12T14:56:08.902 INFO:teuthology.orchestra.run.smithi131.stdout:45 2023-09-12T14:56:08.903 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-09-12T14:56:08.903 DEBUG:teuthology.orchestra.run.smithi131:> sudo stdin-killer -- python3 -c ' 2023-09-12T14:56:08.903 DEBUG:teuthology.orchestra.run.smithi131:> import glob 2023-09-12T14:56:08.904 DEBUG:teuthology.orchestra.run.smithi131:> import re 2023-09-12T14:56:08.904 DEBUG:teuthology.orchestra.run.smithi131:> import os 2023-09-12T14:56:08.904 DEBUG:teuthology.orchestra.run.smithi131:> import subprocess 2023-09-12T14:56:08.905 DEBUG:teuthology.orchestra.run.smithi131:> 2023-09-12T14:56:08.905 DEBUG:teuthology.orchestra.run.smithi131:> def _find_admin_socket(client_name): 2023-09-12T14:56:08.905 DEBUG:teuthology.orchestra.run.smithi131:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-12T14:56:08.906 DEBUG:teuthology.orchestra.run.smithi131:> files = glob.glob(asok_path) 2023-09-12T14:56:08.906 DEBUG:teuthology.orchestra.run.smithi131:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-12T14:56:08.907 DEBUG:teuthology.orchestra.run.smithi131:> 2023-09-12T14:56:08.907 DEBUG:teuthology.orchestra.run.smithi131:> # Given a non-glob path, it better be there 2023-09-12T14:56:08.907 DEBUG:teuthology.orchestra.run.smithi131:> if "*" not in asok_path: 2023-09-12T14:56:08.908 DEBUG:teuthology.orchestra.run.smithi131:> assert(len(files) == 1) 2023-09-12T14:56:08.908 DEBUG:teuthology.orchestra.run.smithi131:> return files[0] 2023-09-12T14:56:08.909 DEBUG:teuthology.orchestra.run.smithi131:> 2023-09-12T14:56:08.909 DEBUG:teuthology.orchestra.run.smithi131:> for f in files: 2023-09-12T14:56:08.909 DEBUG:teuthology.orchestra.run.smithi131:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-12T14:56:08.910 DEBUG:teuthology.orchestra.run.smithi131:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-12T14:56:08.910 DEBUG:teuthology.orchestra.run.smithi131:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-12T14:56:08.911 DEBUG:teuthology.orchestra.run.smithi131:> contents = proc_f.read() 2023-09-12T14:56:08.911 DEBUG:teuthology.orchestra.run.smithi131:> if mountpoint in contents: 2023-09-12T14:56:08.912 DEBUG:teuthology.orchestra.run.smithi131:> return f 2023-09-12T14:56:08.912 DEBUG:teuthology.orchestra.run.smithi131:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-12T14:56:08.912 DEBUG:teuthology.orchestra.run.smithi131:> 2023-09-12T14:56:08.913 DEBUG:teuthology.orchestra.run.smithi131:> print(_find_admin_socket("client.0")) 2023-09-12T14:56:08.913 DEBUG:teuthology.orchestra.run.smithi131:> ' 2023-09-12T14:56:09.030 INFO:teuthology.orchestra.run.smithi131.stdout:/var/run/ceph/ceph-client.0.49189.asok 2023-09-12T14:56:09.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-12T14:56:09.042 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.49189.asok 2023-09-12T14:56:09.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:09.043 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.49189.asok status 2023-09-12T14:56:09.190 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-09-12T14:56:09.191 INFO:teuthology.orchestra.run.smithi131.stdout: "metadata": { 2023-09-12T14:56:09.191 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_sha1": "2110e00747b31a0d73768c9e5229da18b26b8aa0", 2023-09-12T14:56:09.191 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 18.0.0-6088-g2110e007 (2110e00747b31a0d73768c9e5229da18b26b8aa0) reef (dev)", 2023-09-12T14:56:09.191 INFO:teuthology.orchestra.run.smithi131.stdout: "entity_id": "0", 2023-09-12T14:56:09.192 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": "smithi131", 2023-09-12T14:56:09.192 INFO:teuthology.orchestra.run.smithi131.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-12T14:56:09.192 INFO:teuthology.orchestra.run.smithi131.stdout: "pid": "49189", 2023-09-12T14:56:09.192 INFO:teuthology.orchestra.run.smithi131.stdout: "root": "/" 2023-09-12T14:56:09.192 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:56:09.193 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_count": 0, 2023-09-12T14:56:09.193 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_pinned_count": 0, 2023-09-12T14:56:09.193 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 5502, 2023-09-12T14:56:09.193 INFO:teuthology.orchestra.run.smithi131.stdout: "inst": { 2023-09-12T14:56:09.193 INFO:teuthology.orchestra.run.smithi131.stdout: "name": { 2023-09-12T14:56:09.194 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "client", 2023-09-12T14:56:09.194 INFO:teuthology.orchestra.run.smithi131.stdout: "num": 5502 2023-09-12T14:56:09.194 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:56:09.194 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2023-09-12T14:56:09.194 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2023-09-12T14:56:09.195 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2023-09-12T14:56:09.195 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 3031978928 2023-09-12T14:56:09.195 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-09-12T14:56:09.195 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:56:09.195 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2023-09-12T14:56:09.196 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2023-09-12T14:56:09.196 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2023-09-12T14:56:09.196 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 3031978928 2023-09-12T14:56:09.196 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-09-12T14:56:09.196 INFO:teuthology.orchestra.run.smithi131.stdout: "inst_str": "client.5502 192.168.0.1:0/3031978928", 2023-09-12T14:56:09.197 INFO:teuthology.orchestra.run.smithi131.stdout: "addr_str": "192.168.0.1:0/3031978928", 2023-09-12T14:56:09.197 INFO:teuthology.orchestra.run.smithi131.stdout: "inode_count": 1, 2023-09-12T14:56:09.197 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_epoch": 13, 2023-09-12T14:56:09.197 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch": 33, 2023-09-12T14:56:09.197 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch_barrier": 0, 2023-09-12T14:56:09.198 INFO:teuthology.orchestra.run.smithi131.stdout: "blocklisted": false, 2023-09-12T14:56:09.198 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs" 2023-09-12T14:56:09.198 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-09-12T14:56:09.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:09.204 DEBUG:teuthology.orchestra.run.smithi131:> stat --file-system '--printf=%T 2023-09-12T14:56:09.205 DEBUG:teuthology.orchestra.run.smithi131:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-12T14:56:09.212 INFO:teuthology.orchestra.run.smithi131.stdout:fuseblk 2023-09-12T14:56:09.212 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-12T14:56:09.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:09.213 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-12T14:56:09.267 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-12T14:56:09.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.440+0000 7fc276a7e640 1 -- 172.21.15.131:0/512992578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc25c003980 msgr2=0x7fc25c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:09.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.440+0000 7fc276a7e640 1 --2- 172.21.15.131:0/512992578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc25c003980 0x7fc25c003da0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc2700cb030 tx=0x7fc25801cf80 comp rx=0 tx=0).stop 2023-09-12T14:56:09.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.440+0000 7fc27501b640 1 -- 172.21.15.131:0/512992578 reap_dead start 2023-09-12T14:56:09.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.440+0000 7fc276a7e640 1 -- 172.21.15.131:0/512992578 shutdown_connections 2023-09-12T14:56:09.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.440+0000 7fc276a7e640 1 -- 172.21.15.131:0/512992578 >> 172.21.15.131:0/512992578 conn(0x7fc27006ab10 msgr2=0x7fc2700af730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:09.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.440+0000 7fc276a7e640 1 -- 172.21.15.131:0/512992578 shutdown_connections 2023-09-12T14:56:09.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.440+0000 7fc276a7e640 1 -- 172.21.15.131:0/512992578 wait complete. 2023-09-12T14:56:09.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.440+0000 7fc276a7e640 1 Processor -- start 2023-09-12T14:56:09.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.440+0000 7fc276a7e640 1 -- start start 2023-09-12T14:56:09.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.440+0000 7fc276a7e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc25c003980 0x7fc27015fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:09.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.440+0000 7fc276a7e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc27013ae20 0x7fc27013df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:09.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.440+0000 7fc276a7e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc27013e4d0 0x7fc270155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:09.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.440+0000 7fc276a7e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27013aa00 con 0x7fc27013e4d0 2023-09-12T14:56:09.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.440+0000 7fc276a7e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc270160340 con 0x7fc25c003980 2023-09-12T14:56:09.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.440+0000 7fc276a7e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc270139b30 con 0x7fc27013ae20 2023-09-12T14:56:09.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.440+0000 7fc27501b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc27013e4d0 0x7fc270155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:09.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.440+0000 7fc27501b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc27013e4d0 0x7fc270155ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48928/0 (socket says 172.21.15.131:48928) 2023-09-12T14:56:09.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.440+0000 7fc27501b640 1 -- 172.21.15.131:0/3066059896 learned_addr learned my addr 172.21.15.131:0/3066059896 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:09.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.440+0000 7fc27481a640 1 --2- 172.21.15.131:0/3066059896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc25c003980 0x7fc27015fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:09.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.440+0000 7fc26ffff640 1 --2- 172.21.15.131:0/3066059896 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc27013ae20 0x7fc27013df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:09.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.440+0000 7fc27501b640 1 -- 172.21.15.131:0/3066059896 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc27013ae20 msgr2=0x7fc27013df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:09.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.440+0000 7fc27501b640 1 --2- 172.21.15.131:0/3066059896 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc27013ae20 0x7fc27013df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:09.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.440+0000 7fc27501b640 1 -- 172.21.15.131:0/3066059896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc25c003980 msgr2=0x7fc27015fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:09.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.440+0000 7fc27501b640 1 --2- 172.21.15.131:0/3066059896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc25c003980 0x7fc27015fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:09.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.440+0000 7fc27501b640 1 -- 172.21.15.131:0/3066059896 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc270139d80 con 0x7fc27013e4d0 2023-09-12T14:56:09.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.440+0000 7fc26ffff640 1 --2- 172.21.15.131:0/3066059896 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc27013ae20 0x7fc27013df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:09.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.444+0000 7fc27501b640 1 --2- 172.21.15.131:0/3066059896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc27013e4d0 0x7fc270155ea0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc2700b0dc0 tx=0x7fc26800b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:09.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.444+0000 7fc26d7fa640 1 -- 172.21.15.131:0/3066059896 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268011850 con 0x7fc27013e4d0 2023-09-12T14:56:09.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.444+0000 7fc26d7fa640 1 -- 172.21.15.131:0/3066059896 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc2680119d0 con 0x7fc27013e4d0 2023-09-12T14:56:09.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.444+0000 7fc276a7e640 1 -- 172.21.15.131:0/3066059896 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27013a060 con 0x7fc27013e4d0 2023-09-12T14:56:09.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.444+0000 7fc26d7fa640 1 -- 172.21.15.131:0/3066059896 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268011de0 con 0x7fc27013e4d0 2023-09-12T14:56:09.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.444+0000 7fc276a7e640 1 -- 172.21.15.131:0/3066059896 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27013a620 con 0x7fc27013e4d0 2023-09-12T14:56:09.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.444+0000 7fc276a7e640 1 -- 172.21.15.131:0/3066059896 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc27013ab40 con 0x7fc27013e4d0 2023-09-12T14:56:09.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.444+0000 7fc26d7fa640 1 -- 172.21.15.131:0/3066059896 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fc268017070 con 0x7fc27013e4d0 2023-09-12T14:56:09.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.444+0000 7fc26d7fa640 1 --2- 172.21.15.131:0/3066059896 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc250065b20 0x7fc250067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:09.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.444+0000 7fc26d7fa640 1 -- 172.21.15.131:0/3066059896 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc268083ea0 con 0x7fc27013e4d0 2023-09-12T14:56:09.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.444+0000 7fc27481a640 1 --2- 172.21.15.131:0/3066059896 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc250065b20 0x7fc250067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:09.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.444+0000 7fc27481a640 1 --2- 172.21.15.131:0/3066059896 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc250065b20 0x7fc250067fe0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc2580ab000 tx=0x7fc2580a7040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:09.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.448+0000 7fc26d7fa640 1 -- 172.21.15.131:0/3066059896 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fc27013ab40 con 0x7fc27013e4d0 2023-09-12T14:56:09.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.636+0000 7fc276a7e640 1 -- 172.21.15.131:0/3066059896 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc27013ab40 con 0x7fc27013e4d0 2023-09-12T14:56:09.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.640+0000 7fc26d7fa640 1 -- 172.21.15.131:0/3066059896 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc27013ab40 con 0x7fc27013e4d0 2023-09-12T14:56:09.642 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-12T14:56:09.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.640+0000 7fc276a7e640 1 -- 172.21.15.131:0/3066059896 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc250065b20 msgr2=0x7fc250067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:09.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.640+0000 7fc276a7e640 1 --2- 172.21.15.131:0/3066059896 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc250065b20 0x7fc250067fe0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc2580ab000 tx=0x7fc2580a7040 comp rx=0 tx=0).stop 2023-09-12T14:56:09.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.640+0000 7fc276a7e640 1 -- 172.21.15.131:0/3066059896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc27013e4d0 msgr2=0x7fc270155ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:09.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.640+0000 7fc276a7e640 1 --2- 172.21.15.131:0/3066059896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc27013e4d0 0x7fc270155ea0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc2700b0dc0 tx=0x7fc26800b970 comp rx=0 tx=0).stop 2023-09-12T14:56:09.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.640+0000 7fc276a7e640 1 -- 172.21.15.131:0/3066059896 shutdown_connections 2023-09-12T14:56:09.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.640+0000 7fc276a7e640 1 --2- 172.21.15.131:0/3066059896 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc250065b20 0x7fc250067fe0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:09.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.640+0000 7fc276a7e640 1 --2- 172.21.15.131:0/3066059896 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc27013ae20 0x7fc27013df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:09.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.640+0000 7fc276a7e640 1 --2- 172.21.15.131:0/3066059896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc27013e4d0 0x7fc270155ea0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:09.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.640+0000 7fc276a7e640 1 --2- 172.21.15.131:0/3066059896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc25c003980 0x7fc27015fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:09.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.640+0000 7fc276a7e640 1 -- 172.21.15.131:0/3066059896 >> 172.21.15.131:0/3066059896 conn(0x7fc27006ab10 msgr2=0x7fc2700b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:09.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.640+0000 7fc276a7e640 1 -- 172.21.15.131:0/3066059896 shutdown_connections 2023-09-12T14:56:09.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:09.640+0000 7fc276a7e640 1 -- 172.21.15.131:0/3066059896 wait complete. 2023-09-12T14:56:09.660 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-12T14:56:09.660 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-12T14:56:09.661 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi187.front.sepia.ceph.com 2023-09-12T14:56:09.661 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-09-12T14:56:09.661 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-09-12T14:56:09.662 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-12T14:56:09.662 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-12T14:56:09.662 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-12T14:56:09.663 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-09-12T14:56:09.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:09.664 DEBUG:teuthology.orchestra.run.smithi187:> ip addr 2023-09-12T14:56:09.671 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-12T14:56:09.671 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-12T14:56:09.671 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2023-09-12T14:56:09.671 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:56:09.671 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2023-09-12T14:56:09.672 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:56:09.672 INFO:teuthology.orchestra.run.smithi187.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-12T14:56:09.672 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:78 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:56:09.672 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f0 2023-09-12T14:56:09.672 INFO:teuthology.orchestra.run.smithi187.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-12T14:56:09.673 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:79 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:56:09.673 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f1 2023-09-12T14:56:09.673 INFO:teuthology.orchestra.run.smithi187.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-12T14:56:09.673 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b0 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:56:09.673 INFO:teuthology.orchestra.run.smithi187.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-12T14:56:09.674 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b1 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:56:09.674 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.21.15.187/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-09-12T14:56:09.674 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft 42438sec preferred_lft 42438sec 2023-09-12T14:56:09.674 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ec4:7aff:fe88:77b1/64 scope link 2023-09-12T14:56:09.674 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:56:09.674 INFO:teuthology.orchestra.run.smithi187.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-09-12T14:56:09.675 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 02:42:cb:e2:88:30 brd ff:ff:ff:ff:ff:ff 2023-09-12T14:56:09.675 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-09-12T14:56:09.675 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:56:09.675 INFO:teuthology.orchestra.run.smithi187.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-12T14:56:09.675 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-09-12T14:56:09.676 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-12T14:56:09.676 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:56:09.676 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-09-12T14:56:09.676 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:56:09.676 INFO:teuthology.orchestra.run.smithi187.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-12T14:56:09.677 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-09-12T14:56:09.677 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-09-12T14:56:09.677 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:56:09.677 INFO:teuthology.orchestra.run.smithi187.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-12T14:56:09.677 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-09-12T14:56:09.677 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-09-12T14:56:09.678 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2023-09-12T14:56:09.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:09.678 DEBUG:teuthology.orchestra.run.smithi187:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-12T14:56:09.726 INFO:teuthology.orchestra.run.smithi187.stdout:1 2023-09-12T14:56:09.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:09.728 DEBUG:teuthology.orchestra.run.smithi187:> route 2023-09-12T14:56:09.778 INFO:teuthology.orchestra.run.smithi187.stdout:Kernel IP routing table 2023-09-12T14:56:09.778 INFO:teuthology.orchestra.run.smithi187.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-12T14:56:09.778 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-09-12T14:56:09.779 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-12T14:56:09.779 INFO:teuthology.orchestra.run.smithi187.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-09-12T14:56:09.779 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-12T14:56:09.779 INFO:teuthology.orchestra.run.smithi187.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-12T14:56:09.779 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-12T14:56:09.780 INFO:teuthology.orchestra.run.smithi187.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-12T14:56:09.780 INFO:teuthology.orchestra.run.smithi187.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-12T14:56:09.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:09.781 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec bash -c ' 2023-09-12T14:56:09.781 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2023-09-12T14:56:09.781 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-12T14:56:09.782 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-12T14:56:09.782 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-12T14:56:09.782 DEBUG:teuthology.orchestra.run.smithi187:> ') 2023-09-12T14:56:09.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:09.862 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2023-09-12T14:56:09.911 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-09-12T14:56:09.911 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-09-12T14:56:09.912 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-09-12T14:56:09.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:09.912 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set brx.0 up 2023-09-12T14:56:09.968 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-09-12T14:56:09.968 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-12T14:56:09.969 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-09-12T14:56:10.014 INFO:teuthology.orchestra.run.smithi187.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-09-12T14:56:10.015 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-12T14:56:10.015 DEBUG:teuthology.orchestra.run.smithi187:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-09-12T14:56:10.062 DEBUG:teuthology.orchestra.run.smithi187:> sudo modprobe fuse 2023-09-12T14:56:10.120 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-12T14:56:10.121 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-12T14:56:10.173 INFO:teuthology.orchestra.run.smithi187.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-12T14:56:10.174 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-12T14:56:10.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:10.175 DEBUG:teuthology.orchestra.run.smithi187:> ls /sys/fs/fuse/connections 2023-09-12T14:56:10.223 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-12T14:56:10.223 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-09-12T14:56:10.268 DEBUG:teuthology.orchestra.run.smithi187:> sudo modprobe fuse 2023-09-12T14:56:10.282 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-12T14:56:10.283 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-12T14:56:10.342 INFO:teuthology.orchestra.run.smithi187.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-12T14:56:10.343 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-12T14:56:10.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:10.344 DEBUG:teuthology.orchestra.run.smithi187:> ls /sys/fs/fuse/connections 2023-09-12T14:56:10.349 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi187.stderr:2023-09-12T14:56:10.345+0000 7f7ece336580 -1 init, newargv = 0x55b0ab9a9920 newargc=15 2023-09-12T14:56:10.349 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi187.stderr:ceph-fuse[44171]: starting ceph client 2023-09-12T14:56:10.358 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi187.stderr:ceph-fuse[44171]: starting fuse 2023-09-12T14:56:11.454 DEBUG:teuthology.orchestra.run.smithi187:> sudo modprobe fuse 2023-09-12T14:56:11.471 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-12T14:56:11.472 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-12T14:56:11.525 INFO:teuthology.orchestra.run.smithi187.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-12T14:56:11.526 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-12T14:56:11.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:11.528 DEBUG:teuthology.orchestra.run.smithi187:> ls /sys/fs/fuse/connections 2023-09-12T14:56:11.574 INFO:teuthology.orchestra.run.smithi187.stdout:45 2023-09-12T14:56:11.575 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-09-12T14:56:11.575 DEBUG:teuthology.orchestra.run.smithi187:> sudo stdin-killer -- python3 -c ' 2023-09-12T14:56:11.576 DEBUG:teuthology.orchestra.run.smithi187:> import glob 2023-09-12T14:56:11.576 DEBUG:teuthology.orchestra.run.smithi187:> import re 2023-09-12T14:56:11.577 DEBUG:teuthology.orchestra.run.smithi187:> import os 2023-09-12T14:56:11.577 DEBUG:teuthology.orchestra.run.smithi187:> import subprocess 2023-09-12T14:56:11.577 DEBUG:teuthology.orchestra.run.smithi187:> 2023-09-12T14:56:11.578 DEBUG:teuthology.orchestra.run.smithi187:> def _find_admin_socket(client_name): 2023-09-12T14:56:11.578 DEBUG:teuthology.orchestra.run.smithi187:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-09-12T14:56:11.579 DEBUG:teuthology.orchestra.run.smithi187:> files = glob.glob(asok_path) 2023-09-12T14:56:11.579 DEBUG:teuthology.orchestra.run.smithi187:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-09-12T14:56:11.579 DEBUG:teuthology.orchestra.run.smithi187:> 2023-09-12T14:56:11.580 DEBUG:teuthology.orchestra.run.smithi187:> # Given a non-glob path, it better be there 2023-09-12T14:56:11.580 DEBUG:teuthology.orchestra.run.smithi187:> if "*" not in asok_path: 2023-09-12T14:56:11.580 DEBUG:teuthology.orchestra.run.smithi187:> assert(len(files) == 1) 2023-09-12T14:56:11.581 DEBUG:teuthology.orchestra.run.smithi187:> return files[0] 2023-09-12T14:56:11.581 DEBUG:teuthology.orchestra.run.smithi187:> 2023-09-12T14:56:11.582 DEBUG:teuthology.orchestra.run.smithi187:> for f in files: 2023-09-12T14:56:11.582 DEBUG:teuthology.orchestra.run.smithi187:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-12T14:56:11.583 DEBUG:teuthology.orchestra.run.smithi187:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-12T14:56:11.583 DEBUG:teuthology.orchestra.run.smithi187:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-12T14:56:11.583 DEBUG:teuthology.orchestra.run.smithi187:> contents = proc_f.read() 2023-09-12T14:56:11.584 DEBUG:teuthology.orchestra.run.smithi187:> if mountpoint in contents: 2023-09-12T14:56:11.584 DEBUG:teuthology.orchestra.run.smithi187:> return f 2023-09-12T14:56:11.585 DEBUG:teuthology.orchestra.run.smithi187:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-12T14:56:11.585 DEBUG:teuthology.orchestra.run.smithi187:> 2023-09-12T14:56:11.585 DEBUG:teuthology.orchestra.run.smithi187:> print(_find_admin_socket("client.1")) 2023-09-12T14:56:11.585 DEBUG:teuthology.orchestra.run.smithi187:> ' 2023-09-12T14:56:11.700 INFO:teuthology.orchestra.run.smithi187.stdout:/var/run/ceph/ceph-client.1.44171.asok 2023-09-12T14:56:11.703 INFO:teuthology.orchestra.run.smithi187.stderr:2023-09-12T14:56:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-12T14:56:11.711 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.44171.asok 2023-09-12T14:56:11.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:11.712 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.44171.asok status 2023-09-12T14:56:11.863 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2023-09-12T14:56:11.864 INFO:teuthology.orchestra.run.smithi187.stdout: "metadata": { 2023-09-12T14:56:11.864 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_sha1": "2110e00747b31a0d73768c9e5229da18b26b8aa0", 2023-09-12T14:56:11.864 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_version": "ceph version 18.0.0-6088-g2110e007 (2110e00747b31a0d73768c9e5229da18b26b8aa0) reef (dev)", 2023-09-12T14:56:11.864 INFO:teuthology.orchestra.run.smithi187.stdout: "entity_id": "1", 2023-09-12T14:56:11.864 INFO:teuthology.orchestra.run.smithi187.stdout: "hostname": "smithi187", 2023-09-12T14:56:11.865 INFO:teuthology.orchestra.run.smithi187.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-09-12T14:56:11.865 INFO:teuthology.orchestra.run.smithi187.stdout: "pid": "44171", 2023-09-12T14:56:11.865 INFO:teuthology.orchestra.run.smithi187.stdout: "root": "/" 2023-09-12T14:56:11.865 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-09-12T14:56:11.866 INFO:teuthology.orchestra.run.smithi187.stdout: "dentry_count": 0, 2023-09-12T14:56:11.866 INFO:teuthology.orchestra.run.smithi187.stdout: "dentry_pinned_count": 0, 2023-09-12T14:56:11.866 INFO:teuthology.orchestra.run.smithi187.stdout: "id": 4793, 2023-09-12T14:56:11.866 INFO:teuthology.orchestra.run.smithi187.stdout: "inst": { 2023-09-12T14:56:11.866 INFO:teuthology.orchestra.run.smithi187.stdout: "name": { 2023-09-12T14:56:11.867 INFO:teuthology.orchestra.run.smithi187.stdout: "type": "client", 2023-09-12T14:56:11.867 INFO:teuthology.orchestra.run.smithi187.stdout: "num": 4793 2023-09-12T14:56:11.867 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-09-12T14:56:11.867 INFO:teuthology.orchestra.run.smithi187.stdout: "addr": { 2023-09-12T14:56:11.867 INFO:teuthology.orchestra.run.smithi187.stdout: "type": "v1", 2023-09-12T14:56:11.867 INFO:teuthology.orchestra.run.smithi187.stdout: "addr": "192.168.0.1:0", 2023-09-12T14:56:11.868 INFO:teuthology.orchestra.run.smithi187.stdout: "nonce": 2314557369 2023-09-12T14:56:11.868 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-09-12T14:56:11.868 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-09-12T14:56:11.868 INFO:teuthology.orchestra.run.smithi187.stdout: "addr": { 2023-09-12T14:56:11.868 INFO:teuthology.orchestra.run.smithi187.stdout: "type": "v1", 2023-09-12T14:56:11.869 INFO:teuthology.orchestra.run.smithi187.stdout: "addr": "192.168.0.1:0", 2023-09-12T14:56:11.869 INFO:teuthology.orchestra.run.smithi187.stdout: "nonce": 2314557369 2023-09-12T14:56:11.869 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-09-12T14:56:11.869 INFO:teuthology.orchestra.run.smithi187.stdout: "inst_str": "client.4793 192.168.0.1:0/2314557369", 2023-09-12T14:56:11.869 INFO:teuthology.orchestra.run.smithi187.stdout: "addr_str": "192.168.0.1:0/2314557369", 2023-09-12T14:56:11.870 INFO:teuthology.orchestra.run.smithi187.stdout: "inode_count": 1, 2023-09-12T14:56:11.870 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_epoch": 13, 2023-09-12T14:56:11.870 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_epoch": 33, 2023-09-12T14:56:11.870 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_epoch_barrier": 0, 2023-09-12T14:56:11.870 INFO:teuthology.orchestra.run.smithi187.stdout: "blocklisted": false, 2023-09-12T14:56:11.870 INFO:teuthology.orchestra.run.smithi187.stdout: "fs_name": "cephfs" 2023-09-12T14:56:11.871 INFO:teuthology.orchestra.run.smithi187.stdout:} 2023-09-12T14:56:11.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:11.877 DEBUG:teuthology.orchestra.run.smithi187:> stat --file-system '--printf=%T 2023-09-12T14:56:11.878 DEBUG:teuthology.orchestra.run.smithi187:> ' -- /home/ubuntu/cephtest/mnt.1 2023-09-12T14:56:11.883 INFO:teuthology.orchestra.run.smithi187.stdout:fuseblk 2023-09-12T14:56:11.884 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-09-12T14:56:11.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:11.885 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-09-12T14:56:11.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:11.939 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2023-09-12T14:56:11.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:11.957 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quota.max_bytes -v 0 ./subdir) 2023-09-12T14:56:12.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:12.101 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.quota.max_bytes ./subdir) 2023-09-12T14:56:12.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T14:56:12.117 INFO:teuthology.orchestra.run.smithi131.stderr:./subdir: ceph.quota.max_bytes: No such attribute 2023-09-12T14:56:12.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:12.118 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quota.max_bytes -v 1K ./subdir) 2023-09-12T14:56:12.168 INFO:teuthology.orchestra.run.smithi131.stderr:setfattr: ./subdir: Invalid argument 2023-09-12T14:56:12.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T14:56:12.169 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-12T14:56:12.170 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-12T14:56:12.255 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-12T14:56:12.304 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-12T14:56:12.304 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:56:12.305 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-12T14:56:12.311 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:56:12.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.480+0000 7f8ba406e640 1 -- 172.21.15.131:0/1361938419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b9c06d1c0 msgr2=0x7f8b9c06d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:12.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.480+0000 7f8ba406e640 1 --2- 172.21.15.131:0/1361938419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b9c06d1c0 0x7f8b9c06d610 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c070300 tx=0x7f8b8c01c810 comp rx=0 tx=0).stop 2023-09-12T14:56:12.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.480+0000 7f8ba406e640 1 -- 172.21.15.131:0/1361938419 shutdown_connections 2023-09-12T14:56:12.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.480+0000 7f8ba406e640 1 --2- 172.21.15.131:0/1361938419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b9c06d1c0 0x7f8b9c06d610 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:12.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.480+0000 7f8ba406e640 1 --2- 172.21.15.131:0/1361938419 >> v2:172.21.15.187:3300/0 conn(0x7f8b9c153380 0x7f8b9c159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:12.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.480+0000 7f8ba406e640 1 -- 172.21.15.131:0/1361938419 >> 172.21.15.131:0/1361938419 conn(0x7f8b9c06a750 msgr2=0x7f8b9c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:12.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.480+0000 7f8ba406e640 1 -- 172.21.15.131:0/1361938419 shutdown_connections 2023-09-12T14:56:12.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.480+0000 7f8ba406e640 1 -- 172.21.15.131:0/1361938419 wait complete. 2023-09-12T14:56:12.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.480+0000 7f8ba406e640 1 Processor -- start 2023-09-12T14:56:12.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.480+0000 7f8ba406e640 1 -- start start 2023-09-12T14:56:12.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.480+0000 7f8ba406e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b9c06d1c0 0x7f8b9c0c9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:12.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.480+0000 7f8ba406e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b9c153380 0x7f8b9c14ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:12.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.480+0000 7f8ba406e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b9c1596b0 0x7f8b9c142c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:12.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.480+0000 7f8ba406e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c14f260 con 0x7f8b9c06d1c0 2023-09-12T14:56:12.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.480+0000 7f8ba406e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c143140 con 0x7f8b9c1596b0 2023-09-12T14:56:12.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.480+0000 7f8ba406e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c14f500 con 0x7f8b9c153380 2023-09-12T14:56:12.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.480+0000 7f8ba1e0a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b9c06d1c0 0x7f8b9c0c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:12.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.484+0000 7f8ba1e0a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b9c06d1c0 0x7f8b9c0c9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48950/0 (socket says 172.21.15.131:48950) 2023-09-12T14:56:12.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.484+0000 7f8ba1e0a640 1 -- 172.21.15.131:0/3503562013 learned_addr learned my addr 172.21.15.131:0/3503562013 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:12.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.484+0000 7f8ba260b640 1 --2- 172.21.15.131:0/3503562013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b9c1596b0 0x7f8b9c142c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:12.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.484+0000 7f8ba1609640 1 --2- 172.21.15.131:0/3503562013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b9c153380 0x7f8b9c14ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:12.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.484+0000 7f8ba1e0a640 1 -- 172.21.15.131:0/3503562013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b9c153380 msgr2=0x7f8b9c14ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:12.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.484+0000 7f8ba1e0a640 1 --2- 172.21.15.131:0/3503562013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b9c153380 0x7f8b9c14ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:12.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.484+0000 7f8ba1e0a640 1 -- 172.21.15.131:0/3503562013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b9c1596b0 msgr2=0x7f8b9c142c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:12.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.484+0000 7f8ba1e0a640 1 --2- 172.21.15.131:0/3503562013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b9c1596b0 0x7f8b9c142c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:12.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.484+0000 7f8ba1e0a640 1 -- 172.21.15.131:0/3503562013 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c009560 con 0x7f8b9c06d1c0 2023-09-12T14:56:12.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.484+0000 7f8ba1e0a640 1 --2- 172.21.15.131:0/3503562013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b9c06d1c0 0x7f8b9c0c9780 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8b9400cc30 tx=0x7f8b9400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:12.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.484+0000 7f8b9a7fc640 1 -- 172.21.15.131:0/3503562013 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9400f650 con 0x7f8b9c06d1c0 2023-09-12T14:56:12.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.484+0000 7f8b9a7fc640 1 -- 172.21.15.131:0/3503562013 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8b9400f8a0 con 0x7f8b9c06d1c0 2023-09-12T14:56:12.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.484+0000 7f8ba406e640 1 -- 172.21.15.131:0/3503562013 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b9c14f7e0 con 0x7f8b9c06d1c0 2023-09-12T14:56:12.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.484+0000 7f8b9a7fc640 1 -- 172.21.15.131:0/3503562013 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9400fcb0 con 0x7f8b9c06d1c0 2023-09-12T14:56:12.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.484+0000 7f8ba406e640 1 -- 172.21.15.131:0/3503562013 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b9c148380 con 0x7f8b9c06d1c0 2023-09-12T14:56:12.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.484+0000 7f8ba406e640 1 -- 172.21.15.131:0/3503562013 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b9c1485a0 con 0x7f8b9c06d1c0 2023-09-12T14:56:12.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.484+0000 7f8b9a7fc640 1 -- 172.21.15.131:0/3503562013 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f8b94018400 con 0x7f8b9c06d1c0 2023-09-12T14:56:12.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.484+0000 7f8b9a7fc640 1 --2- 172.21.15.131:0/3503562013 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8b88065b20 0x7f8b88067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:12.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.484+0000 7f8ba1609640 1 --2- 172.21.15.131:0/3503562013 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8b88065b20 0x7f8b88067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:12.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.484+0000 7f8b9a7fc640 1 -- 172.21.15.131:0/3503562013 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b94083f50 con 0x7f8b9c06d1c0 2023-09-12T14:56:12.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.484+0000 7f8ba1609640 1 --2- 172.21.15.131:0/3503562013 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8b88065b20 0x7f8b88067fe0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c0aec20 tx=0x7f8b90008040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:12.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.488+0000 7f8b9a7fc640 1 -- 172.21.15.131:0/3503562013 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f8b940518a0 con 0x7f8b9c06d1c0 2023-09-12T14:56:12.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.648+0000 7f8ba406e640 1 -- 172.21.15.131:0/3503562013 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b9c073580 con 0x7f8b9c06d1c0 2023-09-12T14:56:12.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.648+0000 7f8b9a7fc640 1 -- 172.21.15.131:0/3503562013 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f8b9401d090 con 0x7f8b9c06d1c0 2023-09-12T14:56:12.651 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:56:12.652 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":33,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:55:57.898819+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:55:55.034593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-09-12T14:55:55.855927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:56:12.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.652+0000 7f8ba406e640 1 -- 172.21.15.131:0/3503562013 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8b88065b20 msgr2=0x7f8b88067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:12.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.652+0000 7f8ba406e640 1 --2- 172.21.15.131:0/3503562013 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8b88065b20 0x7f8b88067fe0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c0aec20 tx=0x7f8b90008040 comp rx=0 tx=0).stop 2023-09-12T14:56:12.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.652+0000 7f8ba406e640 1 -- 172.21.15.131:0/3503562013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b9c06d1c0 msgr2=0x7f8b9c0c9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:12.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.652+0000 7f8ba406e640 1 --2- 172.21.15.131:0/3503562013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b9c06d1c0 0x7f8b9c0c9780 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8b9400cc30 tx=0x7f8b9400a620 comp rx=0 tx=0).stop 2023-09-12T14:56:12.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.652+0000 7f8ba406e640 1 -- 172.21.15.131:0/3503562013 shutdown_connections 2023-09-12T14:56:12.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.652+0000 7f8ba406e640 1 --2- 172.21.15.131:0/3503562013 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8b88065b20 0x7f8b88067fe0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:12.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.652+0000 7f8ba406e640 1 --2- 172.21.15.131:0/3503562013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b9c153380 0x7f8b9c14ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:12.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.652+0000 7f8ba406e640 1 --2- 172.21.15.131:0/3503562013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b9c1596b0 0x7f8b9c142c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:12.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.652+0000 7f8ba406e640 1 --2- 172.21.15.131:0/3503562013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b9c06d1c0 0x7f8b9c0c9780 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:12.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.652+0000 7f8ba406e640 1 -- 172.21.15.131:0/3503562013 >> 172.21.15.131:0/3503562013 conn(0x7f8b9c06a750 msgr2=0x7f8b9c156d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:12.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.652+0000 7f8ba406e640 1 -- 172.21.15.131:0/3503562013 shutdown_connections 2023-09-12T14:56:12.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.652+0000 7f8ba406e640 1 -- 172.21.15.131:0/3503562013 wait complete. 2023-09-12T14:56:12.670 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T14:54:45.654004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T14:55:55.034593+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T14:55:55.855927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-12T14:56:12.671 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T14:56:12.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.792+0000 7f5bdb420640 1 -- 172.21.15.131:0/3173803023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bc0003fe0 msgr2=0x7f5bc00047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:12.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.792+0000 7f5bdb420640 1 --2- 172.21.15.131:0/3173803023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bc0003fe0 0x7f5bc00047d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40016b0 tx=0x7f5bc401c7a0 comp rx=0 tx=0).stop 2023-09-12T14:56:12.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.792+0000 7f5bd99bd640 1 -- 172.21.15.131:0/3173803023 reap_dead start 2023-09-12T14:56:12.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bdb420640 1 -- 172.21.15.131:0/3173803023 shutdown_connections 2023-09-12T14:56:12.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bdb420640 1 -- 172.21.15.131:0/3173803023 >> 172.21.15.131:0/3173803023 conn(0x7f5bd4069840 msgr2=0x7f5bd40cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:12.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bdb420640 1 -- 172.21.15.131:0/3173803023 shutdown_connections 2023-09-12T14:56:12.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bdb420640 1 -- 172.21.15.131:0/3173803023 wait complete. 2023-09-12T14:56:12.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bdb420640 1 Processor -- start 2023-09-12T14:56:12.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bdb420640 1 -- start start 2023-09-12T14:56:12.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bdb420640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bc0003fe0 0x7f5bd414ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:12.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bdb420640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bd414f270 0x7f5bd41579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:12.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bdb420640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bd414df40 0x7f5bd4157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:12.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bdb420640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd40ae830 con 0x7f5bc0003fe0 2023-09-12T14:56:12.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bdb420640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd40aeaa0 con 0x7f5bd414f270 2023-09-12T14:56:12.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bdb420640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd40aed80 con 0x7f5bd414df40 2023-09-12T14:56:12.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bd91bc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bc0003fe0 0x7f5bd414ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:12.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bd91bc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bc0003fe0 0x7f5bd414ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48962/0 (socket says 172.21.15.131:48962) 2023-09-12T14:56:12.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bd91bc640 1 -- 172.21.15.131:0/3244138026 learned_addr learned my addr 172.21.15.131:0/3244138026 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:12.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bd99bd640 1 --2- 172.21.15.131:0/3244138026 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bd414df40 0x7f5bd4157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:12.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bd89bb640 1 --2- 172.21.15.131:0/3244138026 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bd414f270 0x7f5bd41579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:12.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bd91bc640 1 -- 172.21.15.131:0/3244138026 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bd414df40 msgr2=0x7f5bd4157ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:12.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bd91bc640 1 --2- 172.21.15.131:0/3244138026 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bd414df40 0x7f5bd4157ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:12.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bd91bc640 1 -- 172.21.15.131:0/3244138026 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bd414f270 msgr2=0x7f5bd41579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:12.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bd91bc640 1 --2- 172.21.15.131:0/3244138026 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bd414f270 0x7f5bd41579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:12.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bd91bc640 1 -- 172.21.15.131:0/3244138026 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd40af060 con 0x7f5bc0003fe0 2023-09-12T14:56:12.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bd99bd640 1 --2- 172.21.15.131:0/3244138026 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bd414df40 0x7f5bd4157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:12.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bd89bb640 1 --2- 172.21.15.131:0/3244138026 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bd414f270 0x7f5bd41579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:12.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bd91bc640 1 --2- 172.21.15.131:0/3244138026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bc0003fe0 0x7f5bd414ed30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000fe40 tx=0x7f5bd000c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:12.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bc9ffb640 1 -- 172.21.15.131:0/3244138026 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0018640 con 0x7f5bc0003fe0 2023-09-12T14:56:12.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bc9ffb640 1 -- 172.21.15.131:0/3244138026 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5bd00187a0 con 0x7f5bc0003fe0 2023-09-12T14:56:12.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bc9ffb640 1 -- 172.21.15.131:0/3244138026 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0018b70 con 0x7f5bc0003fe0 2023-09-12T14:56:12.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bdb420640 1 -- 172.21.15.131:0/3244138026 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd414f6f0 con 0x7f5bc0003fe0 2023-09-12T14:56:12.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.796+0000 7f5bdb420640 1 -- 172.21.15.131:0/3244138026 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd41eebb0 con 0x7f5bc0003fe0 2023-09-12T14:56:12.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.800+0000 7f5bdb420640 1 -- 172.21.15.131:0/3244138026 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd4073580 con 0x7f5bc0003fe0 2023-09-12T14:56:12.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.800+0000 7f5bc9ffb640 1 -- 172.21.15.131:0/3244138026 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f5bd00213f0 con 0x7f5bc0003fe0 2023-09-12T14:56:12.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.800+0000 7f5bc9ffb640 1 --2- 172.21.15.131:0/3244138026 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5bb8065b20 0x7f5bb8067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:12.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.800+0000 7f5bc9ffb640 1 -- 172.21.15.131:0/3244138026 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bd0084960 con 0x7f5bc0003fe0 2023-09-12T14:56:12.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.800+0000 7f5bd89bb640 1 --2- 172.21.15.131:0/3244138026 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5bb8065b20 0x7f5bb8067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:12.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.800+0000 7f5bd89bb640 1 --2- 172.21.15.131:0/3244138026 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5bb8065b20 0x7f5bb8067fe0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4001680 tx=0x7f5bc4019540 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:12.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.804+0000 7f5bc9ffb640 1 -- 172.21.15.131:0/3244138026 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f5bd0055d40 con 0x7f5bc0003fe0 2023-09-12T14:56:12.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.960+0000 7f5bdb420640 1 -- 172.21.15.131:0/3244138026 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5bd40af1f0 con 0x7f5bc0003fe0 2023-09-12T14:56:12.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.964+0000 7f5bc9ffb640 1 -- 172.21.15.131:0/3244138026 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5bd00558d0 con 0x7f5bc0003fe0 2023-09-12T14:56:12.966 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-09-12T14:56:12.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.964+0000 7f5bdb420640 1 -- 172.21.15.131:0/3244138026 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5bb8065b20 msgr2=0x7f5bb8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:12.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.964+0000 7f5bdb420640 1 --2- 172.21.15.131:0/3244138026 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5bb8065b20 0x7f5bb8067fe0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4001680 tx=0x7f5bc4019540 comp rx=0 tx=0).stop 2023-09-12T14:56:12.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.964+0000 7f5bdb420640 1 -- 172.21.15.131:0/3244138026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bc0003fe0 msgr2=0x7f5bd414ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:12.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.964+0000 7f5bdb420640 1 --2- 172.21.15.131:0/3244138026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bc0003fe0 0x7f5bd414ed30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000fe40 tx=0x7f5bd000c7a0 comp rx=0 tx=0).stop 2023-09-12T14:56:12.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.964+0000 7f5bdb420640 1 -- 172.21.15.131:0/3244138026 shutdown_connections 2023-09-12T14:56:12.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.964+0000 7f5bdb420640 1 --2- 172.21.15.131:0/3244138026 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5bb8065b20 0x7f5bb8067fe0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:12.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.964+0000 7f5bdb420640 1 --2- 172.21.15.131:0/3244138026 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bd414df40 0x7f5bd4157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:12.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.964+0000 7f5bdb420640 1 --2- 172.21.15.131:0/3244138026 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bd414f270 0x7f5bd41579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:12.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.964+0000 7f5bdb420640 1 --2- 172.21.15.131:0/3244138026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bc0003fe0 0x7f5bd414ed30 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:12.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.964+0000 7f5bdb420640 1 -- 172.21.15.131:0/3244138026 >> 172.21.15.131:0/3244138026 conn(0x7f5bd4069840 msgr2=0x7f5bd40c8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:12.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.964+0000 7f5bdb420640 1 -- 172.21.15.131:0/3244138026 shutdown_connections 2023-09-12T14:56:12.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:12.964+0000 7f5bdb420640 1 -- 172.21.15.131:0/3244138026 wait complete. 2023-09-12T14:56:12.983 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T14:56:13.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.096+0000 7f1bc47bd640 1 -- 172.21.15.131:0/3215606394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bbc156e80 msgr2=0x7f1bbc06eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:13.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.096+0000 7f1bc47bd640 1 --2- 172.21.15.131:0/3215606394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bbc156e80 0x7f1bbc06eca0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0ca160 tx=0x7f1bac01c8c0 comp rx=0 tx=0).stop 2023-09-12T14:56:13.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.096+0000 7f1bc47bd640 1 -- 172.21.15.131:0/3215606394 shutdown_connections 2023-09-12T14:56:13.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.096+0000 7f1bc47bd640 1 --2- 172.21.15.131:0/3215606394 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bbc06baa0 0x7f1bbc1577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:13.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.096+0000 7f1bc47bd640 1 --2- 172.21.15.131:0/3215606394 >> v2:172.21.15.187:3300/0 conn(0x7f1bbc06f1e0 0x7f1bbc157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:13.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.096+0000 7f1bc47bd640 1 --2- 172.21.15.131:0/3215606394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bbc156e80 0x7f1bbc06eca0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:13.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.096+0000 7f1bc47bd640 1 -- 172.21.15.131:0/3215606394 >> 172.21.15.131:0/3215606394 conn(0x7f1bbc069840 msgr2=0x7f1bbc0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:13.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.096+0000 7f1bc47bd640 1 -- 172.21.15.131:0/3215606394 shutdown_connections 2023-09-12T14:56:13.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.096+0000 7f1bc47bd640 1 -- 172.21.15.131:0/3215606394 wait complete. 2023-09-12T14:56:13.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.096+0000 7f1bc47bd640 1 Processor -- start 2023-09-12T14:56:13.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.096+0000 7f1bc47bd640 1 -- start start 2023-09-12T14:56:13.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.096+0000 7f1bc47bd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bbc06baa0 0x7f1bbc1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:13.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.096+0000 7f1bc47bd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bbc06f1e0 0x7f1bbc14c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:13.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.096+0000 7f1bc47bd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bbc156e80 0x7f1bbc14b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:13.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.096+0000 7f1bc47bd640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc14a970 con 0x7f1bbc06f1e0 2023-09-12T14:56:13.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.096+0000 7f1bc47bd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc14b1d0 con 0x7f1bbc156e80 2023-09-12T14:56:13.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.096+0000 7f1bc47bd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc14f3f0 con 0x7f1bbc06baa0 2023-09-12T14:56:13.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.096+0000 7f1bc1d58640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bbc06f1e0 0x7f1bbc14c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:13.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.096+0000 7f1bc1d58640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bbc06f1e0 0x7f1bbc14c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48986/0 (socket says 172.21.15.131:48986) 2023-09-12T14:56:13.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.096+0000 7f1bc1d58640 1 -- 172.21.15.131:0/3988192883 learned_addr learned my addr 172.21.15.131:0/3988192883 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:13.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.096+0000 7f1bc2559640 1 --2- 172.21.15.131:0/3988192883 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bbc06baa0 0x7f1bbc1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:13.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.096+0000 7f1bc2d5a640 1 --2- 172.21.15.131:0/3988192883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bbc156e80 0x7f1bbc14b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:13.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.100+0000 7f1bc1d58640 1 -- 172.21.15.131:0/3988192883 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bbc06baa0 msgr2=0x7f1bbc1393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:13.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.100+0000 7f1bc1d58640 1 --2- 172.21.15.131:0/3988192883 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bbc06baa0 0x7f1bbc1393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:13.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.100+0000 7f1bc1d58640 1 -- 172.21.15.131:0/3988192883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bbc156e80 msgr2=0x7f1bbc14b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:13.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.100+0000 7f1bc1d58640 1 --2- 172.21.15.131:0/3988192883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bbc156e80 0x7f1bbc14b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:13.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.100+0000 7f1bc1d58640 1 -- 172.21.15.131:0/3988192883 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bac009560 con 0x7f1bbc06f1e0 2023-09-12T14:56:13.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.100+0000 7f1bc2d5a640 1 --2- 172.21.15.131:0/3988192883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bbc156e80 0x7f1bbc14b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:13.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.100+0000 7f1bc2559640 1 --2- 172.21.15.131:0/3988192883 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bbc06baa0 0x7f1bbc1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:13.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.100+0000 7f1bc1d58640 1 --2- 172.21.15.131:0/3988192883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bbc06f1e0 0x7f1bbc14c160 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800c9a0 tx=0x7f1bb800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:13.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.100+0000 7f1bb2ffd640 1 -- 172.21.15.131:0/3988192883 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8007bb0 con 0x7f1bbc06f1e0 2023-09-12T14:56:13.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.100+0000 7f1bb2ffd640 1 -- 172.21.15.131:0/3988192883 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1bb8007e00 con 0x7f1bbc06f1e0 2023-09-12T14:56:13.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.100+0000 7f1bb2ffd640 1 -- 172.21.15.131:0/3988192883 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8005280 con 0x7f1bbc06f1e0 2023-09-12T14:56:13.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.100+0000 7f1bc47bd640 1 -- 172.21.15.131:0/3988192883 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bbc14f110 con 0x7f1bbc06f1e0 2023-09-12T14:56:13.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.100+0000 7f1bc47bd640 1 -- 172.21.15.131:0/3988192883 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bbc14ae60 con 0x7f1bbc06f1e0 2023-09-12T14:56:13.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.100+0000 7f1bc47bd640 1 -- 172.21.15.131:0/3988192883 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bbc0cd390 con 0x7f1bbc06f1e0 2023-09-12T14:56:13.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.100+0000 7f1bb2ffd640 1 -- 172.21.15.131:0/3988192883 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f1bb80053e0 con 0x7f1bbc06f1e0 2023-09-12T14:56:13.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.100+0000 7f1bb2ffd640 1 --2- 172.21.15.131:0/3988192883 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1ba0065b20 0x7f1ba0067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:13.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.100+0000 7f1bb2ffd640 1 -- 172.21.15.131:0/3988192883 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1bb80838e0 con 0x7f1bbc06f1e0 2023-09-12T14:56:13.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.100+0000 7f1bc2559640 1 --2- 172.21.15.131:0/3988192883 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1ba0065b20 0x7f1ba0067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:13.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.100+0000 7f1bc2559640 1 --2- 172.21.15.131:0/3988192883 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1ba0065b20 0x7f1ba0067fe0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1bac0af000 tx=0x7f1bac01b100 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:13.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.104+0000 7f1bb2ffd640 1 -- 172.21.15.131:0/3988192883 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f1bb8054cc0 con 0x7f1bbc06f1e0 2023-09-12T14:56:13.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.256+0000 7f1bc47bd640 1 -- 172.21.15.131:0/3988192883 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1bbc073580 con 0x7f1bbc06f1e0 2023-09-12T14:56:13.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.256+0000 7f1bb2ffd640 1 -- 172.21.15.131:0/3988192883 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1bb801d020 con 0x7f1bbc06f1e0 2023-09-12T14:56:13.260 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-09-12T14:56:13.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.260+0000 7f1bc47bd640 1 -- 172.21.15.131:0/3988192883 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1ba0065b20 msgr2=0x7f1ba0067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:13.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.260+0000 7f1bc47bd640 1 --2- 172.21.15.131:0/3988192883 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1ba0065b20 0x7f1ba0067fe0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1bac0af000 tx=0x7f1bac01b100 comp rx=0 tx=0).stop 2023-09-12T14:56:13.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.260+0000 7f1bc47bd640 1 -- 172.21.15.131:0/3988192883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bbc06f1e0 msgr2=0x7f1bbc14c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:13.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.260+0000 7f1bc47bd640 1 --2- 172.21.15.131:0/3988192883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bbc06f1e0 0x7f1bbc14c160 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800c9a0 tx=0x7f1bb800ce60 comp rx=0 tx=0).stop 2023-09-12T14:56:13.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.260+0000 7f1bc47bd640 1 -- 172.21.15.131:0/3988192883 shutdown_connections 2023-09-12T14:56:13.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.260+0000 7f1bc47bd640 1 --2- 172.21.15.131:0/3988192883 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1ba0065b20 0x7f1ba0067fe0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:13.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.260+0000 7f1bc47bd640 1 --2- 172.21.15.131:0/3988192883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bbc156e80 0x7f1bbc14b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:13.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.260+0000 7f1bc47bd640 1 --2- 172.21.15.131:0/3988192883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bbc06f1e0 0x7f1bbc14c160 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:13.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.260+0000 7f1bc47bd640 1 --2- 172.21.15.131:0/3988192883 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bbc06baa0 0x7f1bbc1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:13.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.260+0000 7f1bc47bd640 1 -- 172.21.15.131:0/3988192883 >> 172.21.15.131:0/3988192883 conn(0x7f1bbc069840 msgr2=0x7f1bbc159840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:13.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.260+0000 7f1bc47bd640 1 -- 172.21.15.131:0/3988192883 shutdown_connections 2023-09-12T14:56:13.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.260+0000 7f1bc47bd640 1 -- 172.21.15.131:0/3988192883 wait complete. 2023-09-12T14:56:13.277 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T14:56:13.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d984e5640 1 -- 172.21.15.131:0/2870413056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d90158d70 msgr2=0x7f6d9006b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:13.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d984e5640 1 --2- 172.21.15.131:0/2870413056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d90158d70 0x7f6d9006b430 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6d90070ff0 tx=0x7f6d8c01c810 comp rx=0 tx=0).stop 2023-09-12T14:56:13.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d984e5640 1 -- 172.21.15.131:0/2870413056 shutdown_connections 2023-09-12T14:56:13.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d984e5640 1 --2- 172.21.15.131:0/2870413056 >> v2:172.21.15.187:3300/0 conn(0x7f6d900abae0 0x7f6d900c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:13.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d984e5640 1 --2- 172.21.15.131:0/2870413056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d90158d70 0x7f6d9006b430 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:13.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d984e5640 1 -- 172.21.15.131:0/2870413056 >> 172.21.15.131:0/2870413056 conn(0x7f6d90069840 msgr2=0x7f6d900cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:13.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d984e5640 1 -- 172.21.15.131:0/2870413056 shutdown_connections 2023-09-12T14:56:13.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d984e5640 1 -- 172.21.15.131:0/2870413056 wait complete. 2023-09-12T14:56:13.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d984e5640 1 Processor -- start 2023-09-12T14:56:13.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d984e5640 1 -- start start 2023-09-12T14:56:13.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d984e5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d900abae0 0x7f6d900aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:13.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d984e5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d900cc430 0x7f6d9013f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:13.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d984e5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d90158d70 0x7f6d900a6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:13.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d984e5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d90150db0 con 0x7f6d90158d70 2023-09-12T14:56:13.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d984e5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d900ab220 con 0x7f6d900abae0 2023-09-12T14:56:13.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d984e5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d900ab500 con 0x7f6d900cc430 2023-09-12T14:56:13.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d96a82640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d90158d70 0x7f6d900a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:13.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d96a82640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d90158d70 0x7f6d900a6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48998/0 (socket says 172.21.15.131:48998) 2023-09-12T14:56:13.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d96a82640 1 -- 172.21.15.131:0/1455199613 learned_addr learned my addr 172.21.15.131:0/1455199613 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:13.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d95a80640 1 --2- 172.21.15.131:0/1455199613 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d900cc430 0x7f6d9013f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:13.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d96281640 1 --2- 172.21.15.131:0/1455199613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d900abae0 0x7f6d900aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:13.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d96a82640 1 -- 172.21.15.131:0/1455199613 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d900cc430 msgr2=0x7f6d9013f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:13.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d96a82640 1 --2- 172.21.15.131:0/1455199613 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d900cc430 0x7f6d9013f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:13.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d96a82640 1 -- 172.21.15.131:0/1455199613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d900abae0 msgr2=0x7f6d900aabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:13.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d96a82640 1 --2- 172.21.15.131:0/1455199613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d900abae0 0x7f6d900aabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:13.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d96a82640 1 -- 172.21.15.131:0/1455199613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c009560 con 0x7f6d90158d70 2023-09-12T14:56:13.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d95a80640 1 --2- 172.21.15.131:0/1455199613 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d900cc430 0x7f6d9013f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:13.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d96a82640 1 --2- 172.21.15.131:0/1455199613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d90158d70 0x7f6d900a6620 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6d88001570 tx=0x7f6d8800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:13.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d86ffd640 1 -- 172.21.15.131:0/1455199613 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88003db0 con 0x7f6d90158d70 2023-09-12T14:56:13.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d86ffd640 1 -- 172.21.15.131:0/1455199613 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6d88005070 con 0x7f6d90158d70 2023-09-12T14:56:13.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d984e5640 1 -- 172.21.15.131:0/1455199613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d900ab7e0 con 0x7f6d90158d70 2023-09-12T14:56:13.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d86ffd640 1 -- 172.21.15.131:0/1455199613 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88005480 con 0x7f6d90158d70 2023-09-12T14:56:13.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.404+0000 7f6d984e5640 1 -- 172.21.15.131:0/1455199613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d9013f710 con 0x7f6d90158d70 2023-09-12T14:56:13.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.408+0000 7f6d984e5640 1 -- 172.21.15.131:0/1455199613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d90150f20 con 0x7f6d90158d70 2023-09-12T14:56:13.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.408+0000 7f6d86ffd640 1 -- 172.21.15.131:0/1455199613 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f6d880055e0 con 0x7f6d90158d70 2023-09-12T14:56:13.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.408+0000 7f6d86ffd640 1 --2- 172.21.15.131:0/1455199613 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6d74065b20 0x7f6d74067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:13.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.408+0000 7f6d86ffd640 1 -- 172.21.15.131:0/1455199613 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d88082e60 con 0x7f6d90158d70 2023-09-12T14:56:13.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.408+0000 7f6d96281640 1 --2- 172.21.15.131:0/1455199613 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6d74065b20 0x7f6d74067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:13.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.408+0000 7f6d96281640 1 --2- 172.21.15.131:0/1455199613 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6d74065b20 0x7f6d74067fe0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0af000 tx=0x7f6d8c01b050 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:13.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.412+0000 7f6d86ffd640 1 -- 172.21.15.131:0/1455199613 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f6d880507a0 con 0x7f6d90158d70 2023-09-12T14:56:13.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.564+0000 7f6d984e5640 1 -- 172.21.15.131:0/1455199613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6d90073580 con 0x7f6d90158d70 2023-09-12T14:56:13.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.564+0000 7f6d86ffd640 1 -- 172.21.15.131:0/1455199613 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6d88017020 con 0x7f6d90158d70 2023-09-12T14:56:13.568 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-09-12T14:56:13.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.564+0000 7f6d984e5640 1 -- 172.21.15.131:0/1455199613 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6d74065b20 msgr2=0x7f6d74067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:13.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.568+0000 7f6d984e5640 1 --2- 172.21.15.131:0/1455199613 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6d74065b20 0x7f6d74067fe0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0af000 tx=0x7f6d8c01b050 comp rx=0 tx=0).stop 2023-09-12T14:56:13.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.568+0000 7f6d984e5640 1 -- 172.21.15.131:0/1455199613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d90158d70 msgr2=0x7f6d900a6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:13.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.568+0000 7f6d984e5640 1 --2- 172.21.15.131:0/1455199613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d90158d70 0x7f6d900a6620 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6d88001570 tx=0x7f6d8800cd20 comp rx=0 tx=0).stop 2023-09-12T14:56:13.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.568+0000 7f6d984e5640 1 -- 172.21.15.131:0/1455199613 shutdown_connections 2023-09-12T14:56:13.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.568+0000 7f6d984e5640 1 --2- 172.21.15.131:0/1455199613 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6d74065b20 0x7f6d74067fe0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:13.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.568+0000 7f6d984e5640 1 --2- 172.21.15.131:0/1455199613 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d900cc430 0x7f6d9013f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:13.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.568+0000 7f6d984e5640 1 --2- 172.21.15.131:0/1455199613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d90158d70 0x7f6d900a6620 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:13.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.568+0000 7f6d984e5640 1 --2- 172.21.15.131:0/1455199613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d900abae0 0x7f6d900aabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:13.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.568+0000 7f6d984e5640 1 -- 172.21.15.131:0/1455199613 >> 172.21.15.131:0/1455199613 conn(0x7f6d90069840 msgr2=0x7f6d900adbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:13.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.568+0000 7f6d984e5640 1 -- 172.21.15.131:0/1455199613 shutdown_connections 2023-09-12T14:56:13.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.568+0000 7f6d984e5640 1 -- 172.21.15.131:0/1455199613 wait complete. 2023-09-12T14:56:13.585 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-12T14:56:13.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.696+0000 7f8b1c766640 1 -- 172.21.15.131:0/3683289324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b140cf2d0 msgr2=0x7f8b1415e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:13.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.696+0000 7f8b1c766640 1 --2- 172.21.15.131:0/3683289324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b140cf2d0 0x7f8b1415e2e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8b14089db0 tx=0x7f8b0c01cd60 comp rx=0 tx=0).stop 2023-09-12T14:56:13.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.696+0000 7f8b1c766640 1 -- 172.21.15.131:0/3683289324 shutdown_connections 2023-09-12T14:56:13.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.696+0000 7f8b1c766640 1 --2- 172.21.15.131:0/3683289324 >> v2:172.21.15.187:3300/0 conn(0x7f8b140d1680 0x7f8b1415e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:13.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.696+0000 7f8b1c766640 1 --2- 172.21.15.131:0/3683289324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b140cf2d0 0x7f8b1415e2e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:13.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.696+0000 7f8b1c766640 1 -- 172.21.15.131:0/3683289324 >> 172.21.15.131:0/3683289324 conn(0x7f8b1409a200 msgr2=0x7f8b140d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:13.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.696+0000 7f8b1c766640 1 -- 172.21.15.131:0/3683289324 shutdown_connections 2023-09-12T14:56:13.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.696+0000 7f8b1c766640 1 -- 172.21.15.131:0/3683289324 wait complete. 2023-09-12T14:56:13.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.696+0000 7f8b1c766640 1 Processor -- start 2023-09-12T14:56:13.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.696+0000 7f8b1c766640 1 -- start start 2023-09-12T14:56:13.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.696+0000 7f8b1c766640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b140cf2d0 0x7f8b1415b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:13.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.696+0000 7f8b1c766640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b140cf6b0 0x7f8b140b5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:13.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.696+0000 7f8b1c766640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b140d1680 0x7f8b14152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:13.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.696+0000 7f8b1c766640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b14153bc0 con 0x7f8b140cf2d0 2023-09-12T14:56:13.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.696+0000 7f8b1c766640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b14153ea0 con 0x7f8b140d1680 2023-09-12T14:56:13.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.696+0000 7f8b1c766640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b141582a0 con 0x7f8b140cf6b0 2023-09-12T14:56:13.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.696+0000 7f8b1a502640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b140cf2d0 0x7f8b1415b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:13.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.696+0000 7f8b1a502640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b140cf2d0 0x7f8b1415b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49022/0 (socket says 172.21.15.131:49022) 2023-09-12T14:56:13.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.696+0000 7f8b1a502640 1 -- 172.21.15.131:0/138440332 learned_addr learned my addr 172.21.15.131:0/138440332 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:13.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.696+0000 7f8b1a502640 1 -- 172.21.15.131:0/138440332 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b140cf6b0 msgr2=0x7f8b140b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:13.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.696+0000 7f8b1a502640 1 --2- 172.21.15.131:0/138440332 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b140cf6b0 0x7f8b140b5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:13.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.696+0000 7f8b1a502640 1 -- 172.21.15.131:0/138440332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b140d1680 msgr2=0x7f8b14152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:13.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.696+0000 7f8b1a502640 1 --2- 172.21.15.131:0/138440332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b140d1680 0x7f8b14152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:13.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.696+0000 7f8b1a502640 1 -- 172.21.15.131:0/138440332 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0c0098e0 con 0x7f8b140cf2d0 2023-09-12T14:56:13.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.700+0000 7f8b1a502640 1 --2- 172.21.15.131:0/138440332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b140cf2d0 0x7f8b1415b810 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c019970 tx=0x7f8b0c0ae6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:13.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.700+0000 7f8b06ffd640 1 -- 172.21.15.131:0/138440332 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c0ae8a0 con 0x7f8b140cf2d0 2023-09-12T14:56:13.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.700+0000 7f8b06ffd640 1 -- 172.21.15.131:0/138440332 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8b0c0aeaf0 con 0x7f8b140cf2d0 2023-09-12T14:56:13.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.700+0000 7f8b1c766640 1 -- 172.21.15.131:0/138440332 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b14153890 con 0x7f8b140cf2d0 2023-09-12T14:56:13.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.700+0000 7f8b06ffd640 1 -- 172.21.15.131:0/138440332 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c004000 con 0x7f8b140cf2d0 2023-09-12T14:56:13.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.700+0000 7f8b1c766640 1 -- 172.21.15.131:0/138440332 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b140b6050 con 0x7f8b140cf2d0 2023-09-12T14:56:13.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.700+0000 7f8b1c766640 1 -- 172.21.15.131:0/138440332 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b14073580 con 0x7f8b140cf2d0 2023-09-12T14:56:13.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.700+0000 7f8b06ffd640 1 -- 172.21.15.131:0/138440332 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f8b0c0b4040 con 0x7f8b140cf2d0 2023-09-12T14:56:13.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.700+0000 7f8b06ffd640 1 --2- 172.21.15.131:0/138440332 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8af8065b20 0x7f8af8067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:13.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.700+0000 7f8b06ffd640 1 -- 172.21.15.131:0/138440332 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b0c11e6d0 con 0x7f8b140cf2d0 2023-09-12T14:56:13.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.700+0000 7f8b19d01640 1 --2- 172.21.15.131:0/138440332 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8af8065b20 0x7f8af8067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:13.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.700+0000 7f8b19d01640 1 --2- 172.21.15.131:0/138440332 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8af8065b20 0x7f8af8067fe0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8b080058d0 tx=0x7f8b08005860 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:13.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.704+0000 7f8b06ffd640 1 -- 172.21.15.131:0/138440332 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f8b0c0ec0b0 con 0x7f8b140cf2d0 2023-09-12T14:56:13.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.856+0000 7f8b1c766640 1 -- 172.21.15.131:0/138440332 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8b14153d00 con 0x7f8b140cf2d0 2023-09-12T14:56:13.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.856+0000 7f8b06ffd640 1 -- 172.21.15.131:0/138440332 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8b14153d00 con 0x7f8b140cf2d0 2023-09-12T14:56:13.861 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2023-09-12T14:56:13.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.860+0000 7f8b1c766640 1 -- 172.21.15.131:0/138440332 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8af8065b20 msgr2=0x7f8af8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:13.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.860+0000 7f8b1c766640 1 --2- 172.21.15.131:0/138440332 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8af8065b20 0x7f8af8067fe0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8b080058d0 tx=0x7f8b08005860 comp rx=0 tx=0).stop 2023-09-12T14:56:13.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.860+0000 7f8b1c766640 1 -- 172.21.15.131:0/138440332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b140cf2d0 msgr2=0x7f8b1415b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:13.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.860+0000 7f8b1c766640 1 --2- 172.21.15.131:0/138440332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b140cf2d0 0x7f8b1415b810 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c019970 tx=0x7f8b0c0ae6b0 comp rx=0 tx=0).stop 2023-09-12T14:56:13.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.860+0000 7f8b1c766640 1 -- 172.21.15.131:0/138440332 shutdown_connections 2023-09-12T14:56:13.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.860+0000 7f8b1c766640 1 --2- 172.21.15.131:0/138440332 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8af8065b20 0x7f8af8067fe0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:13.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.860+0000 7f8b1c766640 1 --2- 172.21.15.131:0/138440332 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b140cf6b0 0x7f8b140b5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:13.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.860+0000 7f8b1c766640 1 --2- 172.21.15.131:0/138440332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b140d1680 0x7f8b14152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:13.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.860+0000 7f8b1c766640 1 --2- 172.21.15.131:0/138440332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b140cf2d0 0x7f8b1415b810 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:13.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.860+0000 7f8b1c766640 1 -- 172.21.15.131:0/138440332 >> 172.21.15.131:0/138440332 conn(0x7f8b1409a200 msgr2=0x7f8b140cc340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:13.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.860+0000 7f8b1c766640 1 -- 172.21.15.131:0/138440332 shutdown_connections 2023-09-12T14:56:13.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:13.860+0000 7f8b1c766640 1 -- 172.21.15.131:0/138440332 wait complete. 2023-09-12T14:56:13.877 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi131.front.sepia.ceph.com... 2023-09-12T14:56:13.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:13.878 DEBUG:teuthology.orchestra.run.smithi131:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-12T14:56:13.886 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[49189]: fuse finished with error 0 and tester_r 0 2023-09-12T14:56:13.888 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2023-09-12T14:56:13.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:13.889 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-12T14:56:14.050 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:56:14.051 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-12T14:56:14.058 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:56:14.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.216+0000 7f6d00009640 1 -- 172.21.15.131:0/4290120415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cf8143800 msgr2=0x7f6cf8144070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:14.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.216+0000 7f6d00009640 1 --2- 172.21.15.131:0/4290120415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cf8143800 0x7f6cf8144070 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009900 tx=0x7f6cf401c7c0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.216+0000 7f6d00009640 1 -- 172.21.15.131:0/4290120415 shutdown_connections 2023-09-12T14:56:14.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.216+0000 7f6d00009640 1 --2- 172.21.15.131:0/4290120415 >> v2:172.21.15.187:3300/0 conn(0x7f6cf806f1e0 0x7f6cf806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.216+0000 7f6d00009640 1 --2- 172.21.15.131:0/4290120415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf8143be0 0x7f6cf806eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.216+0000 7f6d00009640 1 --2- 172.21.15.131:0/4290120415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cf8143800 0x7f6cf8144070 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.216+0000 7f6d00009640 1 -- 172.21.15.131:0/4290120415 >> 172.21.15.131:0/4290120415 conn(0x7f6cf8069840 msgr2=0x7f6cf80cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:14.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.216+0000 7f6d00009640 1 -- 172.21.15.131:0/4290120415 shutdown_connections 2023-09-12T14:56:14.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.216+0000 7f6d00009640 1 -- 172.21.15.131:0/4290120415 wait complete. 2023-09-12T14:56:14.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.220+0000 7f6d00009640 1 Processor -- start 2023-09-12T14:56:14.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.220+0000 7f6d00009640 1 -- start start 2023-09-12T14:56:14.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.220+0000 7f6d00009640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf806f1e0 0x7f6cf8135480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:14.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.220+0000 7f6d00009640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cf8143800 0x7f6cf815dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:14.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.220+0000 7f6d00009640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cf8143be0 0x7f6cf813a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:14.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.220+0000 7f6d00009640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf81359c0 con 0x7f6cf8143be0 2023-09-12T14:56:14.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.220+0000 7f6d00009640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf813a1d0 con 0x7f6cf8143800 2023-09-12T14:56:14.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.220+0000 7f6d00009640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf81395c0 con 0x7f6cf806f1e0 2023-09-12T14:56:14.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.220+0000 7f6cfe5a6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cf8143be0 0x7f6cf813a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:14.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.220+0000 7f6cfe5a6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cf8143be0 0x7f6cf813a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49042/0 (socket says 172.21.15.131:49042) 2023-09-12T14:56:14.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.220+0000 7f6cfe5a6640 1 -- 172.21.15.131:0/2009461188 learned_addr learned my addr 172.21.15.131:0/2009461188 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:14.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.220+0000 7f6cfdda5640 1 --2- 172.21.15.131:0/2009461188 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf806f1e0 0x7f6cf8135480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:14.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.220+0000 7f6cfd5a4640 1 --2- 172.21.15.131:0/2009461188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cf8143800 0x7f6cf815dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:14.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.220+0000 7f6cfe5a6640 1 -- 172.21.15.131:0/2009461188 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf806f1e0 msgr2=0x7f6cf8135480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:14.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.220+0000 7f6cfe5a6640 1 --2- 172.21.15.131:0/2009461188 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf806f1e0 0x7f6cf8135480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.220+0000 7f6cfe5a6640 1 -- 172.21.15.131:0/2009461188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cf8143800 msgr2=0x7f6cf815dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:14.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.220+0000 7f6cfe5a6640 1 --2- 172.21.15.131:0/2009461188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cf8143800 0x7f6cf815dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.220+0000 7f6cfe5a6640 1 -- 172.21.15.131:0/2009461188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf4009560 con 0x7f6cf8143be0 2023-09-12T14:56:14.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.220+0000 7f6cfe5a6640 1 --2- 172.21.15.131:0/2009461188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cf8143be0 0x7f6cf813a660 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6cf000ccb0 tx=0x7f6cf000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:14.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.220+0000 7f6cee7fc640 1 -- 172.21.15.131:0/2009461188 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0004b30 con 0x7f6cf8143be0 2023-09-12T14:56:14.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.220+0000 7f6cee7fc640 1 -- 172.21.15.131:0/2009461188 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6cf0004da0 con 0x7f6cf8143be0 2023-09-12T14:56:14.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.220+0000 7f6d00009640 1 -- 172.21.15.131:0/2009461188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf81398a0 con 0x7f6cf8143be0 2023-09-12T14:56:14.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.220+0000 7f6cee7fc640 1 -- 172.21.15.131:0/2009461188 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0005280 con 0x7f6cf8143be0 2023-09-12T14:56:14.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.220+0000 7f6d00009640 1 -- 172.21.15.131:0/2009461188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf8139e60 con 0x7f6cf8143be0 2023-09-12T14:56:14.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.220+0000 7f6d00009640 1 -- 172.21.15.131:0/2009461188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf8073580 con 0x7f6cf8143be0 2023-09-12T14:56:14.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.224+0000 7f6cee7fc640 1 -- 172.21.15.131:0/2009461188 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f6cf00053e0 con 0x7f6cf8143be0 2023-09-12T14:56:14.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.224+0000 7f6cee7fc640 1 --2- 172.21.15.131:0/2009461188 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6cdc065b20 0x7f6cdc067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:14.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.224+0000 7f6cee7fc640 1 -- 172.21.15.131:0/2009461188 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6cf0083bf0 con 0x7f6cf8143be0 2023-09-12T14:56:14.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.224+0000 7f6cfdda5640 1 --2- 172.21.15.131:0/2009461188 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6cdc065b20 0x7f6cdc067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:14.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.224+0000 7f6cfdda5640 1 --2- 172.21.15.131:0/2009461188 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6cdc065b20 0x7f6cdc067fe0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40af000 tx=0x7f6cf401b000 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:14.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.228+0000 7f6cee7fc640 1 -- 172.21.15.131:0/2009461188 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f6cf0051540 con 0x7f6cf8143be0 2023-09-12T14:56:14.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.372+0000 7f6d00009640 1 -- 172.21.15.131:0/2009461188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6cf80ab6b0 con 0x7f6cf8143be0 2023-09-12T14:56:14.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.372+0000 7f6cee7fc640 1 -- 172.21.15.131:0/2009461188 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f6cf001d020 con 0x7f6cf8143be0 2023-09-12T14:56:14.375 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:56:14.375 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":33,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:55:57.898819+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:55:55.034593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-09-12T14:55:55.855927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:56:14.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.372+0000 7f6d00009640 1 -- 172.21.15.131:0/2009461188 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6cdc065b20 msgr2=0x7f6cdc067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:14.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.372+0000 7f6d00009640 1 --2- 172.21.15.131:0/2009461188 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6cdc065b20 0x7f6cdc067fe0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40af000 tx=0x7f6cf401b000 comp rx=0 tx=0).stop 2023-09-12T14:56:14.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.376+0000 7f6d00009640 1 -- 172.21.15.131:0/2009461188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cf8143be0 msgr2=0x7f6cf813a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:14.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.376+0000 7f6d00009640 1 --2- 172.21.15.131:0/2009461188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cf8143be0 0x7f6cf813a660 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6cf000ccb0 tx=0x7f6cf000a620 comp rx=0 tx=0).stop 2023-09-12T14:56:14.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.376+0000 7f6d00009640 1 -- 172.21.15.131:0/2009461188 shutdown_connections 2023-09-12T14:56:14.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.376+0000 7f6d00009640 1 --2- 172.21.15.131:0/2009461188 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6cdc065b20 0x7f6cdc067fe0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.379 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.376+0000 7f6d00009640 1 --2- 172.21.15.131:0/2009461188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cf8143be0 0x7f6cf813a660 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.379 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.376+0000 7f6d00009640 1 --2- 172.21.15.131:0/2009461188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cf8143800 0x7f6cf815dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.379 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.376+0000 7f6d00009640 1 --2- 172.21.15.131:0/2009461188 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf806f1e0 0x7f6cf8135480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.379 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.376+0000 7f6d00009640 1 -- 172.21.15.131:0/2009461188 >> 172.21.15.131:0/2009461188 conn(0x7f6cf8069840 msgr2=0x7f6cf813fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:14.379 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.376+0000 7f6d00009640 1 -- 172.21.15.131:0/2009461188 shutdown_connections 2023-09-12T14:56:14.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.376+0000 7f6d00009640 1 -- 172.21.15.131:0/2009461188 wait complete. 2023-09-12T14:56:14.393 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T14:54:45.654004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T14:55:55.034593+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T14:55:55.855927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-12T14:56:14.394 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T14:56:14.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.512+0000 7f9c0ab93640 1 -- 172.21.15.131:0/2382427319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c04071280 msgr2=0x7f9c040c8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:14.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.512+0000 7f9c0ab93640 1 --2- 172.21.15.131:0/2382427319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c04071280 0x7f9c040c8c90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009cb0 tx=0x7f9bf401ccc0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.512+0000 7f9c0ab93640 1 -- 172.21.15.131:0/2382427319 shutdown_connections 2023-09-12T14:56:14.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.512+0000 7f9c0ab93640 1 --2- 172.21.15.131:0/2382427319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c04071280 0x7f9c040c8c90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.512+0000 7f9c0ab93640 1 --2- 172.21.15.131:0/2382427319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c041567d0 0x7f9c04069240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.512+0000 7f9c0ab93640 1 -- 172.21.15.131:0/2382427319 >> 172.21.15.131:0/2382427319 conn(0x7f9c0406b310 msgr2=0x7f9c0406b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:14.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.512+0000 7f9c0ab93640 1 -- 172.21.15.131:0/2382427319 shutdown_connections 2023-09-12T14:56:14.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.512+0000 7f9c0ab93640 1 -- 172.21.15.131:0/2382427319 wait complete. 2023-09-12T14:56:14.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.516+0000 7f9c0ab93640 1 Processor -- start 2023-09-12T14:56:14.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.516+0000 7f9c0ab93640 1 -- start start 2023-09-12T14:56:14.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.516+0000 7f9c0ab93640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c0406d860 0x7f9c0415b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:14.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.516+0000 7f9c0ab93640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c04071280 0x7f9c041493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:14.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.516+0000 7f9c0ab93640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c041567d0 0x7f9c041498e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:14.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.516+0000 7f9c0ab93640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0414f070 con 0x7f9c041567d0 2023-09-12T14:56:14.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.516+0000 7f9c0ab93640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0414e9d0 con 0x7f9c04071280 2023-09-12T14:56:14.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.516+0000 7f9c0ab93640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0415bee0 con 0x7f9c0406d860 2023-09-12T14:56:14.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.516+0000 7f9c09130640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c041567d0 0x7f9c041498e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:14.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.516+0000 7f9c09130640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c041567d0 0x7f9c041498e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49050/0 (socket says 172.21.15.131:49050) 2023-09-12T14:56:14.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.516+0000 7f9c09130640 1 -- 172.21.15.131:0/1433629298 learned_addr learned my addr 172.21.15.131:0/1433629298 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:14.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.516+0000 7f9c03fff640 1 --2- 172.21.15.131:0/1433629298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c04071280 0x7f9c041493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:14.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.516+0000 7f9c0892f640 1 --2- 172.21.15.131:0/1433629298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c0406d860 0x7f9c0415b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:14.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.516+0000 7f9c09130640 1 -- 172.21.15.131:0/1433629298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c0406d860 msgr2=0x7f9c0415b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:14.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.516+0000 7f9c09130640 1 --2- 172.21.15.131:0/1433629298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c0406d860 0x7f9c0415b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.516+0000 7f9c09130640 1 -- 172.21.15.131:0/1433629298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c04071280 msgr2=0x7f9c041493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:14.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.516+0000 7f9c09130640 1 --2- 172.21.15.131:0/1433629298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c04071280 0x7f9c041493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.516+0000 7f9c09130640 1 -- 172.21.15.131:0/1433629298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf40098e0 con 0x7f9c041567d0 2023-09-12T14:56:14.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.516+0000 7f9c09130640 1 --2- 172.21.15.131:0/1433629298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c041567d0 0x7f9c041498e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009bd0 tx=0x7f9bf4003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:14.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.516+0000 7f9c017fa640 1 -- 172.21.15.131:0/1433629298 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4004c50 con 0x7f9c041567d0 2023-09-12T14:56:14.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.516+0000 7f9c017fa640 1 -- 172.21.15.131:0/1433629298 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9bf4004db0 con 0x7f9c041567d0 2023-09-12T14:56:14.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.516+0000 7f9c0ab93640 1 -- 172.21.15.131:0/1433629298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c0415c1c0 con 0x7f9c041567d0 2023-09-12T14:56:14.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.516+0000 7f9c017fa640 1 -- 172.21.15.131:0/1433629298 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4004210 con 0x7f9c041567d0 2023-09-12T14:56:14.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.516+0000 7f9c0ab93640 1 -- 172.21.15.131:0/1433629298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c04150120 con 0x7f9c041567d0 2023-09-12T14:56:14.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.516+0000 7f9c0ab93640 1 -- 172.21.15.131:0/1433629298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c0414f1b0 con 0x7f9c041567d0 2023-09-12T14:56:14.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.516+0000 7f9c017fa640 1 -- 172.21.15.131:0/1433629298 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f9bf40b4040 con 0x7f9c041567d0 2023-09-12T14:56:14.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.520+0000 7f9c017fa640 1 --2- 172.21.15.131:0/1433629298 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9be4065b50 0x7f9be4068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:14.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.520+0000 7f9c017fa640 1 -- 172.21.15.131:0/1433629298 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9bf411dc00 con 0x7f9c041567d0 2023-09-12T14:56:14.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.520+0000 7f9c0892f640 1 --2- 172.21.15.131:0/1433629298 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9be4065b50 0x7f9be4068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:14.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.520+0000 7f9c0892f640 1 --2- 172.21.15.131:0/1433629298 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9be4065b50 0x7f9be4068010 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009ad0 tx=0x7f9bfc0091e0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:14.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.524+0000 7f9c017fa640 1 -- 172.21.15.131:0/1433629298 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f9bf4124080 con 0x7f9c041567d0 2023-09-12T14:56:14.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.672+0000 7f9c0ab93640 1 -- 172.21.15.131:0/1433629298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9c04073580 con 0x7f9c041567d0 2023-09-12T14:56:14.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.676+0000 7f9c017fa640 1 -- 172.21.15.131:0/1433629298 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9bf40b20a0 con 0x7f9c041567d0 2023-09-12T14:56:14.678 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-09-12T14:56:14.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.676+0000 7f9c0ab93640 1 -- 172.21.15.131:0/1433629298 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9be4065b50 msgr2=0x7f9be4068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:14.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.676+0000 7f9c0ab93640 1 --2- 172.21.15.131:0/1433629298 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9be4065b50 0x7f9be4068010 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009ad0 tx=0x7f9bfc0091e0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.676+0000 7f9c0ab93640 1 -- 172.21.15.131:0/1433629298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c041567d0 msgr2=0x7f9c041498e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:14.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.676+0000 7f9c0ab93640 1 --2- 172.21.15.131:0/1433629298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c041567d0 0x7f9c041498e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009bd0 tx=0x7f9bf4003f60 comp rx=0 tx=0).stop 2023-09-12T14:56:14.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.676+0000 7f9c0ab93640 1 -- 172.21.15.131:0/1433629298 shutdown_connections 2023-09-12T14:56:14.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.676+0000 7f9c0ab93640 1 --2- 172.21.15.131:0/1433629298 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f9be4065b50 0x7f9be4068010 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.676+0000 7f9c0ab93640 1 --2- 172.21.15.131:0/1433629298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c041567d0 0x7f9c041498e0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.676+0000 7f9c0ab93640 1 --2- 172.21.15.131:0/1433629298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c04071280 0x7f9c041493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.676+0000 7f9c0ab93640 1 --2- 172.21.15.131:0/1433629298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c0406d860 0x7f9c0415b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.676+0000 7f9c0ab93640 1 -- 172.21.15.131:0/1433629298 >> 172.21.15.131:0/1433629298 conn(0x7f9c0406b310 msgr2=0x7f9c040ae2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:14.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.676+0000 7f9c0ab93640 1 -- 172.21.15.131:0/1433629298 shutdown_connections 2023-09-12T14:56:14.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.676+0000 7f9c0ab93640 1 -- 172.21.15.131:0/1433629298 wait complete. 2023-09-12T14:56:14.697 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T14:56:14.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60b8fed640 1 -- 172.21.15.131:0/534700427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60b40ab730 msgr2=0x7f60b40abb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:14.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60b8fed640 1 --2- 172.21.15.131:0/534700427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60b40ab730 0x7f60b40abb10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f60b40cd8f0 tx=0x7f60a401c830 comp rx=0 tx=0).stop 2023-09-12T14:56:14.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60b8fed640 1 -- 172.21.15.131:0/534700427 shutdown_connections 2023-09-12T14:56:14.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60b8fed640 1 --2- 172.21.15.131:0/534700427 >> v2:172.21.15.187:3300/0 conn(0x7f60b406b050 0x7f60b406b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60b8fed640 1 --2- 172.21.15.131:0/534700427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60b40ab730 0x7f60b40abb10 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60b8fed640 1 -- 172.21.15.131:0/534700427 >> 172.21.15.131:0/534700427 conn(0x7f60b4069840 msgr2=0x7f60b40cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:14.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60b8fed640 1 -- 172.21.15.131:0/534700427 shutdown_connections 2023-09-12T14:56:14.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60b8fed640 1 -- 172.21.15.131:0/534700427 wait complete. 2023-09-12T14:56:14.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60b8fed640 1 Processor -- start 2023-09-12T14:56:14.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60b8fed640 1 -- start start 2023-09-12T14:56:14.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60b8fed640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60b406b050 0x7f60b4145f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:14.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60b8fed640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60b40ab730 0x7f60b415aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:14.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60b8fed640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60b4158180 0x7f60b41456f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:14.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60b8fed640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b40c9890 con 0x7f60b4158180 2023-09-12T14:56:14.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60b8fed640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b40c9b70 con 0x7f60b406b050 2023-09-12T14:56:14.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60b8fed640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60b40c9e50 con 0x7f60b40ab730 2023-09-12T14:56:14.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60b2d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60b4158180 0x7f60b41456f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:14.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60b2d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60b4158180 0x7f60b41456f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49058/0 (socket says 172.21.15.131:49058) 2023-09-12T14:56:14.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60b2d9d640 1 -- 172.21.15.131:0/3576744672 learned_addr learned my addr 172.21.15.131:0/3576744672 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:14.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60b259c640 1 --2- 172.21.15.131:0/3576744672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60b406b050 0x7f60b4145f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:14.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60abfff640 1 --2- 172.21.15.131:0/3576744672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60b40ab730 0x7f60b415aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:14.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60b2d9d640 1 -- 172.21.15.131:0/3576744672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60b40ab730 msgr2=0x7f60b415aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:14.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60b2d9d640 1 --2- 172.21.15.131:0/3576744672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60b40ab730 0x7f60b415aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60b2d9d640 1 -- 172.21.15.131:0/3576744672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60b406b050 msgr2=0x7f60b4145f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:14.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60b2d9d640 1 --2- 172.21.15.131:0/3576744672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60b406b050 0x7f60b4145f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60b2d9d640 1 -- 172.21.15.131:0/3576744672 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a4009560 con 0x7f60b4158180 2023-09-12T14:56:14.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60abfff640 1 --2- 172.21.15.131:0/3576744672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60b40ab730 0x7f60b415aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:14.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60b259c640 1 --2- 172.21.15.131:0/3576744672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60b406b050 0x7f60b4145f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:56:14.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.816+0000 7f60b2d9d640 1 --2- 172.21.15.131:0/3576744672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60b4158180 0x7f60b41456f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00cce0 tx=0x7f60ac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:14.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.820+0000 7f60aaffd640 1 -- 172.21.15.131:0/3576744672 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac00f450 con 0x7f60b4158180 2023-09-12T14:56:14.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.820+0000 7f60aaffd640 1 -- 172.21.15.131:0/3576744672 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f60ac00f6c0 con 0x7f60b4158180 2023-09-12T14:56:14.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.820+0000 7f60b8fed640 1 -- 172.21.15.131:0/3576744672 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b40ca130 con 0x7f60b4158180 2023-09-12T14:56:14.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.820+0000 7f60aaffd640 1 -- 172.21.15.131:0/3576744672 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac00fad0 con 0x7f60b4158180 2023-09-12T14:56:14.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.820+0000 7f60b8fed640 1 -- 172.21.15.131:0/3576744672 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b4146490 con 0x7f60b4158180 2023-09-12T14:56:14.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.820+0000 7f60b8fed640 1 -- 172.21.15.131:0/3576744672 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60b4073580 con 0x7f60b4158180 2023-09-12T14:56:14.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.820+0000 7f60aaffd640 1 -- 172.21.15.131:0/3576744672 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f60ac00fc30 con 0x7f60b4158180 2023-09-12T14:56:14.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.820+0000 7f60aaffd640 1 --2- 172.21.15.131:0/3576744672 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6094065b20 0x7f6094067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:14.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.820+0000 7f60aaffd640 1 -- 172.21.15.131:0/3576744672 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f60ac083f90 con 0x7f60b4158180 2023-09-12T14:56:14.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.820+0000 7f60b259c640 1 --2- 172.21.15.131:0/3576744672 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6094065b20 0x7f6094067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:14.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.820+0000 7f60b259c640 1 --2- 172.21.15.131:0/3576744672 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6094065b20 0x7f6094067fe0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f60a40af000 tx=0x7f60a401b070 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:14.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.824+0000 7f60aaffd640 1 -- 172.21.15.131:0/3576744672 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f60ac0b4db0 con 0x7f60b4158180 2023-09-12T14:56:14.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.976+0000 7f60b8fed640 1 -- 172.21.15.131:0/3576744672 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f60b414de40 con 0x7f60b4158180 2023-09-12T14:56:14.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.976+0000 7f60aaffd640 1 -- 172.21.15.131:0/3576744672 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7f60ac051900 con 0x7f60b4158180 2023-09-12T14:56:14.981 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-09-12T14:56:14.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.980+0000 7f60b8fed640 1 -- 172.21.15.131:0/3576744672 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6094065b20 msgr2=0x7f6094067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:14.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.980+0000 7f60b8fed640 1 --2- 172.21.15.131:0/3576744672 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6094065b20 0x7f6094067fe0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f60a40af000 tx=0x7f60a401b070 comp rx=0 tx=0).stop 2023-09-12T14:56:14.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.980+0000 7f60b8fed640 1 -- 172.21.15.131:0/3576744672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60b4158180 msgr2=0x7f60b41456f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:14.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.980+0000 7f60b8fed640 1 --2- 172.21.15.131:0/3576744672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60b4158180 0x7f60b41456f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00cce0 tx=0x7f60ac00a620 comp rx=0 tx=0).stop 2023-09-12T14:56:14.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.980+0000 7f60b8fed640 1 -- 172.21.15.131:0/3576744672 shutdown_connections 2023-09-12T14:56:14.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.980+0000 7f60b8fed640 1 --2- 172.21.15.131:0/3576744672 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6094065b20 0x7f6094067fe0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.980+0000 7f60b8fed640 1 --2- 172.21.15.131:0/3576744672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60b40ab730 0x7f60b415aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.980+0000 7f60b8fed640 1 --2- 172.21.15.131:0/3576744672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60b4158180 0x7f60b41456f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.980+0000 7f60b8fed640 1 --2- 172.21.15.131:0/3576744672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60b406b050 0x7f60b4145f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:14.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.980+0000 7f60b8fed640 1 -- 172.21.15.131:0/3576744672 >> 172.21.15.131:0/3576744672 conn(0x7f60b4069840 msgr2=0x7f60b40acb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:14.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.980+0000 7f60b8fed640 1 -- 172.21.15.131:0/3576744672 shutdown_connections 2023-09-12T14:56:14.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:14.980+0000 7f60b8fed640 1 -- 172.21.15.131:0/3576744672 wait complete. 2023-09-12T14:56:15.000 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T14:56:15.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.128+0000 7fa2b848a640 1 -- 172.21.15.131:0/4821259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2b01556f0 msgr2=0x7fa2b0158080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:15.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.128+0000 7fa2b848a640 1 --2- 172.21.15.131:0/4821259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2b01556f0 0x7fa2b0158080 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8009900 tx=0x7fa2a801c890 comp rx=0 tx=0).stop 2023-09-12T14:56:15.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.128+0000 7fa2b848a640 1 -- 172.21.15.131:0/4821259 shutdown_connections 2023-09-12T14:56:15.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.128+0000 7fa2b848a640 1 --2- 172.21.15.131:0/4821259 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2b00cb010 0x7fa2b00ab730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:15.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.128+0000 7fa2b848a640 1 --2- 172.21.15.131:0/4821259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2b01556f0 0x7fa2b0158080 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:15.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.128+0000 7fa2b848a640 1 -- 172.21.15.131:0/4821259 >> 172.21.15.131:0/4821259 conn(0x7fa2b0069840 msgr2=0x7fa2b00cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:15.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.128+0000 7fa2b848a640 1 -- 172.21.15.131:0/4821259 shutdown_connections 2023-09-12T14:56:15.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.128+0000 7fa2b848a640 1 -- 172.21.15.131:0/4821259 wait complete. 2023-09-12T14:56:15.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.128+0000 7fa2b848a640 1 Processor -- start 2023-09-12T14:56:15.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.128+0000 7fa2b848a640 1 -- start start 2023-09-12T14:56:15.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.128+0000 7fa2b848a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2b00abc70 0x7fa2b0146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:15.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.128+0000 7fa2b848a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2b00cb010 0x7fa2b0153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:15.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.128+0000 7fa2b848a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2b01556f0 0x7fa2b0143300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:15.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.128+0000 7fa2b848a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b0146640 con 0x7fa2b01556f0 2023-09-12T14:56:15.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.128+0000 7fa2b848a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b01473a0 con 0x7fa2b00abc70 2023-09-12T14:56:15.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.128+0000 7fa2b848a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b0146820 con 0x7fa2b00cb010 2023-09-12T14:56:15.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.128+0000 7fa2b6a27640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2b01556f0 0x7fa2b0143300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:15.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.128+0000 7fa2b6a27640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2b01556f0 0x7fa2b0143300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49088/0 (socket says 172.21.15.131:49088) 2023-09-12T14:56:15.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.128+0000 7fa2b6a27640 1 -- 172.21.15.131:0/2966976713 learned_addr learned my addr 172.21.15.131:0/2966976713 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:15.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.128+0000 7fa2b6226640 1 --2- 172.21.15.131:0/2966976713 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2b00abc70 0x7fa2b0146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:15.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.128+0000 7fa2b5a25640 1 --2- 172.21.15.131:0/2966976713 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2b00cb010 0x7fa2b0153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:15.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.128+0000 7fa2b6a27640 1 -- 172.21.15.131:0/2966976713 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2b00cb010 msgr2=0x7fa2b0153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:15.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.128+0000 7fa2b6a27640 1 --2- 172.21.15.131:0/2966976713 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2b00cb010 0x7fa2b0153420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:15.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.128+0000 7fa2b6a27640 1 -- 172.21.15.131:0/2966976713 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2b00abc70 msgr2=0x7fa2b0146100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:15.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.128+0000 7fa2b6a27640 1 --2- 172.21.15.131:0/2966976713 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2b00abc70 0x7fa2b0146100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:15.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.128+0000 7fa2b6a27640 1 -- 172.21.15.131:0/2966976713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a8009560 con 0x7fa2b01556f0 2023-09-12T14:56:15.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.128+0000 7fa2b6226640 1 --2- 172.21.15.131:0/2966976713 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2b00abc70 0x7fa2b0146100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:56:15.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.132+0000 7fa2b6a27640 1 --2- 172.21.15.131:0/2966976713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2b01556f0 0x7fa2b0143300 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000c770 tx=0x7fa2a000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:15.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.132+0000 7fa2aeffd640 1 -- 172.21.15.131:0/2966976713 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0003b80 con 0x7fa2b01556f0 2023-09-12T14:56:15.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.132+0000 7fa2aeffd640 1 -- 172.21.15.131:0/2966976713 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa2a0003ce0 con 0x7fa2b01556f0 2023-09-12T14:56:15.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.132+0000 7fa2b848a640 1 -- 172.21.15.131:0/2966976713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b0146b00 con 0x7fa2b01556f0 2023-09-12T14:56:15.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.132+0000 7fa2aeffd640 1 -- 172.21.15.131:0/2966976713 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a000aab0 con 0x7fa2b01556f0 2023-09-12T14:56:15.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.132+0000 7fa2b848a640 1 -- 172.21.15.131:0/2966976713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b01470c0 con 0x7fa2b01556f0 2023-09-12T14:56:15.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.132+0000 7fa2b848a640 1 -- 172.21.15.131:0/2966976713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b0073580 con 0x7fa2b01556f0 2023-09-12T14:56:15.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.132+0000 7fa2aeffd640 1 -- 172.21.15.131:0/2966976713 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fa2a000ac10 con 0x7fa2b01556f0 2023-09-12T14:56:15.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.132+0000 7fa2aeffd640 1 --2- 172.21.15.131:0/2966976713 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa29c065b50 0x7fa29c068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:15.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.132+0000 7fa2b6226640 1 --2- 172.21.15.131:0/2966976713 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa29c065b50 0x7fa29c068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:15.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.132+0000 7fa2aeffd640 1 -- 172.21.15.131:0/2966976713 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2a0083160 con 0x7fa2b01556f0 2023-09-12T14:56:15.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.132+0000 7fa2b6226640 1 --2- 172.21.15.131:0/2966976713 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa29c065b50 0x7fa29c068010 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80af000 tx=0x7fa2a801b0d0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:15.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.136+0000 7fa2aeffd640 1 -- 172.21.15.131:0/2966976713 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fa2a0050a20 con 0x7fa2b01556f0 2023-09-12T14:56:15.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.288+0000 7fa2b848a640 1 -- 172.21.15.131:0/2966976713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa2b00cc090 con 0x7fa2b01556f0 2023-09-12T14:56:15.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.288+0000 7fa2aeffd640 1 -- 172.21.15.131:0/2966976713 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa2b00cc090 con 0x7fa2b01556f0 2023-09-12T14:56:15.293 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-09-12T14:56:15.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.292+0000 7fa2b848a640 1 -- 172.21.15.131:0/2966976713 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa29c065b50 msgr2=0x7fa29c068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:15.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.292+0000 7fa2b848a640 1 --2- 172.21.15.131:0/2966976713 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa29c065b50 0x7fa29c068010 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80af000 tx=0x7fa2a801b0d0 comp rx=0 tx=0).stop 2023-09-12T14:56:15.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.292+0000 7fa2b848a640 1 -- 172.21.15.131:0/2966976713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2b01556f0 msgr2=0x7fa2b0143300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:15.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.292+0000 7fa2b848a640 1 --2- 172.21.15.131:0/2966976713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2b01556f0 0x7fa2b0143300 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000c770 tx=0x7fa2a000cc30 comp rx=0 tx=0).stop 2023-09-12T14:56:15.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.292+0000 7fa2b848a640 1 -- 172.21.15.131:0/2966976713 shutdown_connections 2023-09-12T14:56:15.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.292+0000 7fa2b848a640 1 --2- 172.21.15.131:0/2966976713 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa29c065b50 0x7fa29c068010 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:15.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.292+0000 7fa2b848a640 1 --2- 172.21.15.131:0/2966976713 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2b00cb010 0x7fa2b0153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:15.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.292+0000 7fa2b848a640 1 --2- 172.21.15.131:0/2966976713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2b01556f0 0x7fa2b0143300 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:15.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.292+0000 7fa2b848a640 1 --2- 172.21.15.131:0/2966976713 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2b00abc70 0x7fa2b0146100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:15.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.292+0000 7fa2b848a640 1 -- 172.21.15.131:0/2966976713 >> 172.21.15.131:0/2966976713 conn(0x7fa2b0069840 msgr2=0x7fa2b01548b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:15.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.292+0000 7fa2b848a640 1 -- 172.21.15.131:0/2966976713 shutdown_connections 2023-09-12T14:56:15.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.292+0000 7fa2b848a640 1 -- 172.21.15.131:0/2966976713 wait complete. 2023-09-12T14:56:15.312 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-12T14:56:15.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.432+0000 7fb3a759e640 1 -- 172.21.15.131:0/318869672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3a8157120 msgr2=0x7fb3a80afe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:15.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.432+0000 7fb3a759e640 1 --2- 172.21.15.131:0/318869672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3a8157120 0x7fb3a80afe70 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0009c80 tx=0x7fb3a001cd40 comp rx=0 tx=0).stop 2023-09-12T14:56:15.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.432+0000 7fb3a759e640 1 -- 172.21.15.131:0/318869672 shutdown_connections 2023-09-12T14:56:15.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.432+0000 7fb3a759e640 1 --2- 172.21.15.131:0/318869672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3a8157120 0x7fb3a80afe70 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:15.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.432+0000 7fb3a759e640 1 -- 172.21.15.131:0/318869672 >> 172.21.15.131:0/318869672 conn(0x7fb3a8069240 msgr2=0x7fb3a8069650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:15.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.432+0000 7fb3a759e640 1 -- 172.21.15.131:0/318869672 shutdown_connections 2023-09-12T14:56:15.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.432+0000 7fb3a759e640 1 -- 172.21.15.131:0/318869672 wait complete. 2023-09-12T14:56:15.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.432+0000 7fb3a759e640 1 Processor -- start 2023-09-12T14:56:15.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.432+0000 7fb3a759e640 1 -- start start 2023-09-12T14:56:15.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.432+0000 7fb3a759e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3a80aeed0 0x7fb3a8152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:15.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.432+0000 7fb3a759e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3a8157120 0x7fb3a81417b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:15.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.432+0000 7fb3a759e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3a814d700 0x7fb3a814dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:15.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.432+0000 7fb3a759e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a814e0f0 con 0x7fb3a8157120 2023-09-12T14:56:15.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.432+0000 7fb3a759e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a81528c0 con 0x7fb3a814d700 2023-09-12T14:56:15.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.432+0000 7fb3a759e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a8140a30 con 0x7fb3a80aeed0 2023-09-12T14:56:15.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.432+0000 7fb3a5d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3a8157120 0x7fb3a81417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:15.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.432+0000 7fb3a5d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3a8157120 0x7fb3a81417b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49108/0 (socket says 172.21.15.131:49108) 2023-09-12T14:56:15.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.432+0000 7fb3a5d9b640 1 -- 172.21.15.131:0/3637260377 learned_addr learned my addr 172.21.15.131:0/3637260377 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:15.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.436+0000 7fb3a6d9d640 1 --2- 172.21.15.131:0/3637260377 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3a814d700 0x7fb3a814dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:15.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.436+0000 7fb3a659c640 1 --2- 172.21.15.131:0/3637260377 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3a80aeed0 0x7fb3a8152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:15.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.436+0000 7fb3a5d9b640 1 -- 172.21.15.131:0/3637260377 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3a80aeed0 msgr2=0x7fb3a8152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:15.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.436+0000 7fb3a5d9b640 1 --2- 172.21.15.131:0/3637260377 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3a80aeed0 0x7fb3a8152380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:15.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.436+0000 7fb3a5d9b640 1 -- 172.21.15.131:0/3637260377 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3a814d700 msgr2=0x7fb3a814dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:15.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.436+0000 7fb3a5d9b640 1 --2- 172.21.15.131:0/3637260377 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3a814d700 0x7fb3a814dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:15.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.436+0000 7fb3a5d9b640 1 -- 172.21.15.131:0/3637260377 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a00098e0 con 0x7fb3a8157120 2023-09-12T14:56:15.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.436+0000 7fb3a5d9b640 1 --2- 172.21.15.131:0/3637260377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3a8157120 0x7fb3a81417b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb39c00c450 tx=0x7fb39c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:15.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.436+0000 7fb396ffd640 1 -- 172.21.15.131:0/3637260377 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39c012830 con 0x7fb3a8157120 2023-09-12T14:56:15.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.436+0000 7fb396ffd640 1 -- 172.21.15.131:0/3637260377 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb39c012aa0 con 0x7fb3a8157120 2023-09-12T14:56:15.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.436+0000 7fb3a759e640 1 -- 172.21.15.131:0/3637260377 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3a812f7c0 con 0x7fb3a8157120 2023-09-12T14:56:15.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.436+0000 7fb396ffd640 1 -- 172.21.15.131:0/3637260377 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39c011380 con 0x7fb3a8157120 2023-09-12T14:56:15.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.436+0000 7fb3a759e640 1 -- 172.21.15.131:0/3637260377 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3a8140fb0 con 0x7fb3a8157120 2023-09-12T14:56:15.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.436+0000 7fb3a759e640 1 -- 172.21.15.131:0/3637260377 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3a804e110 con 0x7fb3a8157120 2023-09-12T14:56:15.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.436+0000 7fb396ffd640 1 -- 172.21.15.131:0/3637260377 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fb39c0114e0 con 0x7fb3a8157120 2023-09-12T14:56:15.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.436+0000 7fb396ffd640 1 --2- 172.21.15.131:0/3637260377 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb388065b20 0x7fb388067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:15.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.436+0000 7fb3a659c640 1 --2- 172.21.15.131:0/3637260377 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb388065b20 0x7fb388067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:15.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.436+0000 7fb396ffd640 1 -- 172.21.15.131:0/3637260377 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb39c083380 con 0x7fb3a8157120 2023-09-12T14:56:15.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.436+0000 7fb3a659c640 1 --2- 172.21.15.131:0/3637260377 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb388065b20 0x7fb388067fe0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb398000f70 tx=0x7fb398009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:15.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.440+0000 7fb396ffd640 1 -- 172.21.15.131:0/3637260377 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fb39c050d60 con 0x7fb3a8157120 2023-09-12T14:56:15.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.592+0000 7fb3a759e640 1 -- 172.21.15.131:0/3637260377 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb3a80ca290 con 0x7fb3a8157120 2023-09-12T14:56:15.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.592+0000 7fb396ffd640 1 -- 172.21.15.131:0/3637260377 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb39c054380 con 0x7fb3a8157120 2023-09-12T14:56:15.596 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2023-09-12T14:56:15.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.596+0000 7fb3a759e640 1 -- 172.21.15.131:0/3637260377 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb388065b20 msgr2=0x7fb388067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:15.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.596+0000 7fb3a759e640 1 --2- 172.21.15.131:0/3637260377 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb388065b20 0x7fb388067fe0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb398000f70 tx=0x7fb398009040 comp rx=0 tx=0).stop 2023-09-12T14:56:15.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.596+0000 7fb3a759e640 1 -- 172.21.15.131:0/3637260377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3a8157120 msgr2=0x7fb3a81417b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:15.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.596+0000 7fb3a759e640 1 --2- 172.21.15.131:0/3637260377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3a8157120 0x7fb3a81417b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb39c00c450 tx=0x7fb39c00c910 comp rx=0 tx=0).stop 2023-09-12T14:56:15.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.596+0000 7fb3a759e640 1 -- 172.21.15.131:0/3637260377 shutdown_connections 2023-09-12T14:56:15.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.596+0000 7fb3a759e640 1 --2- 172.21.15.131:0/3637260377 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb388065b20 0x7fb388067fe0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:15.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.596+0000 7fb3a759e640 1 --2- 172.21.15.131:0/3637260377 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3a814d700 0x7fb3a814dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:15.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.596+0000 7fb3a759e640 1 --2- 172.21.15.131:0/3637260377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3a8157120 0x7fb3a81417b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:15.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.596+0000 7fb3a759e640 1 --2- 172.21.15.131:0/3637260377 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3a80aeed0 0x7fb3a8152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:15.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.596+0000 7fb3a759e640 1 -- 172.21.15.131:0/3637260377 >> 172.21.15.131:0/3637260377 conn(0x7fb3a8069240 msgr2=0x7fb3a814bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:15.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.596+0000 7fb3a759e640 1 -- 172.21.15.131:0/3637260377 shutdown_connections 2023-09-12T14:56:15.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.596+0000 7fb3a759e640 1 -- 172.21.15.131:0/3637260377 wait complete. 2023-09-12T14:56:15.614 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi187.front.sepia.ceph.com... 2023-09-12T14:56:15.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:15.615 DEBUG:teuthology.orchestra.run.smithi187:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-09-12T14:56:15.629 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi187.stderr:ceph-fuse[44171]: fuse finished with error 0 and tester_r 0 2023-09-12T14:56:15.631 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2023-09-12T14:56:15.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:15.632 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-09-12T14:56:15.744 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:56:15.745 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-12T14:56:15.752 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18514 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-09-12T14:56:15.753 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2023-09-12T14:56:15.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:15.753 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-09-12T14:56:15.797 INFO:teuthology.orchestra.run.smithi131.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-09-12T14:56:15.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T14:56:15.799 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:56:15.799 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-12T14:56:15.805 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19313 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-09-12T14:56:15.806 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2023-09-12T14:56:15.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:15.807 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-09-12T14:56:15.856 INFO:teuthology.orchestra.run.smithi187.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-09-12T14:56:15.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T14:56:15.858 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T14:56:15.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.988+0000 7fb653433640 1 -- 172.21.15.131:0/3508243962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb64c06f390 msgr2=0x7fb64c06f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:15.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.988+0000 7fb653433640 1 --2- 172.21.15.131:0/3508243962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb64c06f390 0x7fb64c06f820 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb64c06eea0 tx=0x7fb64401cd30 comp rx=0 tx=0).stop 2023-09-12T14:56:15.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.988+0000 7fb653433640 1 -- 172.21.15.131:0/3508243962 shutdown_connections 2023-09-12T14:56:15.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.988+0000 7fb653433640 1 --2- 172.21.15.131:0/3508243962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb64c06f390 0x7fb64c06f820 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:15.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.988+0000 7fb653433640 1 --2- 172.21.15.131:0/3508243962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb64c158180 0x7fb64c0cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:15.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.988+0000 7fb653433640 1 --2- 172.21.15.131:0/3508243962 >> v2:172.21.15.187:3300/0 conn(0x7fb64c0ab730 0x7fb64c0abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:15.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.988+0000 7fb653433640 1 -- 172.21.15.131:0/3508243962 >> 172.21.15.131:0/3508243962 conn(0x7fb64c069840 msgr2=0x7fb64c0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:15.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.988+0000 7fb653433640 1 -- 172.21.15.131:0/3508243962 shutdown_connections 2023-09-12T14:56:15.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.988+0000 7fb653433640 1 -- 172.21.15.131:0/3508243962 wait complete. 2023-09-12T14:56:15.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.988+0000 7fb653433640 1 Processor -- start 2023-09-12T14:56:15.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.988+0000 7fb653433640 1 -- start start 2023-09-12T14:56:15.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.988+0000 7fb653433640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb64c06f390 0x7fb64c14c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:15.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.988+0000 7fb653433640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb64c0ab730 0x7fb64c14f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:15.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.988+0000 7fb653433640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb64c158180 0x7fb64c13e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:15.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.988+0000 7fb653433640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c1507b0 con 0x7fb64c0ab730 2023-09-12T14:56:15.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.988+0000 7fb653433640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c150bd0 con 0x7fb64c06f390 2023-09-12T14:56:15.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.988+0000 7fb653433640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c0b2520 con 0x7fb64c158180 2023-09-12T14:56:15.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.988+0000 7fb6509ce640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb64c0ab730 0x7fb64c14f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:15.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.988+0000 7fb6509ce640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb64c0ab730 0x7fb64c14f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49136/0 (socket says 172.21.15.131:49136) 2023-09-12T14:56:15.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.988+0000 7fb6509ce640 1 -- 172.21.15.131:0/3628681908 learned_addr learned my addr 172.21.15.131:0/3628681908 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:15.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.988+0000 7fb6511cf640 1 --2- 172.21.15.131:0/3628681908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb64c06f390 0x7fb64c14c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:15.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.988+0000 7fb6519d0640 1 --2- 172.21.15.131:0/3628681908 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb64c158180 0x7fb64c13e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:16.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.988+0000 7fb6509ce640 1 -- 172.21.15.131:0/3628681908 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb64c158180 msgr2=0x7fb64c13e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:16.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.992+0000 7fb6509ce640 1 --2- 172.21.15.131:0/3628681908 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb64c158180 0x7fb64c13e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.992+0000 7fb6509ce640 1 -- 172.21.15.131:0/3628681908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb64c06f390 msgr2=0x7fb64c14c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:16.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.992+0000 7fb6509ce640 1 --2- 172.21.15.131:0/3628681908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb64c06f390 0x7fb64c14c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.992+0000 7fb6509ce640 1 -- 172.21.15.131:0/3628681908 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6440098e0 con 0x7fb64c0ab730 2023-09-12T14:56:16.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.992+0000 7fb6509ce640 1 --2- 172.21.15.131:0/3628681908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb64c0ab730 0x7fb64c14f4a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00e7c0 tx=0x7fb63c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:16.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.992+0000 7fb641ffb640 1 -- 172.21.15.131:0/3628681908 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c0095b0 con 0x7fb64c0ab730 2023-09-12T14:56:16.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.992+0000 7fb641ffb640 1 -- 172.21.15.131:0/3628681908 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb63c00eed0 con 0x7fb64c0ab730 2023-09-12T14:56:16.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.992+0000 7fb653433640 1 -- 172.21.15.131:0/3628681908 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb64c14fb30 con 0x7fb64c0ab730 2023-09-12T14:56:16.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.992+0000 7fb641ffb640 1 -- 172.21.15.131:0/3628681908 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c005020 con 0x7fb64c0ab730 2023-09-12T14:56:16.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.992+0000 7fb653433640 1 -- 172.21.15.131:0/3628681908 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb64c1500f0 con 0x7fb64c0ab730 2023-09-12T14:56:16.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.992+0000 7fb653433640 1 -- 172.21.15.131:0/3628681908 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb64c1508f0 con 0x7fb64c0ab730 2023-09-12T14:56:16.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.992+0000 7fb641ffb640 1 -- 172.21.15.131:0/3628681908 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fb63c005180 con 0x7fb64c0ab730 2023-09-12T14:56:16.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.992+0000 7fb641ffb640 1 --2- 172.21.15.131:0/3628681908 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb630065b20 0x7fb630067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:16.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.992+0000 7fb641ffb640 1 -- 172.21.15.131:0/3628681908 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb63c014070 con 0x7fb64c0ab730 2023-09-12T14:56:16.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.992+0000 7fb6511cf640 1 --2- 172.21.15.131:0/3628681908 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb630065b20 0x7fb630067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:16.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.992+0000 7fb6511cf640 1 --2- 172.21.15.131:0/3628681908 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb630065b20 0x7fb630067fe0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb64c152a80 tx=0x7fb6480091e0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:16.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:15.996+0000 7fb641ffb640 1 -- 172.21.15.131:0/3628681908 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fb63c051950 con 0x7fb64c0ab730 2023-09-12T14:56:16.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.176+0000 7fb653433640 1 -- 172.21.15.131:0/3628681908 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb64c1508f0 con 0x7fb64c0ab730 2023-09-12T14:56:16.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.176+0000 7fb641ffb640 1 -- 172.21.15.131:0/3628681908 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fb64c1508f0 con 0x7fb64c0ab730 2023-09-12T14:56:16.180 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:56:16.181 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4676,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/3890238785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3890238785},{"type":"v1","addr":"172.21.15.187:6834","nonce":3890238785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4978,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1436257801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":1436257801},{"type":"v1","addr":"172.21.15.187:6835","nonce":1436257801}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4990,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/806466310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":806466310},{"type":"v1","addr":"172.21.15.131:6837","nonce":806466310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-12T14:55:56.870775+0000","modified":"2023-09-12T14:55:58.869201+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/871788191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":871788191},{"type":"v1","addr":"172.21.15.131:6835","nonce":871788191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-09-12T14:56:16.181 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 13 2023-09-12T14:56:16.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.180+0000 7fb653433640 1 -- 172.21.15.131:0/3628681908 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb630065b20 msgr2=0x7fb630067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:16.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.180+0000 7fb653433640 1 --2- 172.21.15.131:0/3628681908 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb630065b20 0x7fb630067fe0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb64c152a80 tx=0x7fb6480091e0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.180+0000 7fb653433640 1 -- 172.21.15.131:0/3628681908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb64c0ab730 msgr2=0x7fb64c14f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:16.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.180+0000 7fb653433640 1 --2- 172.21.15.131:0/3628681908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb64c0ab730 0x7fb64c14f4a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00e7c0 tx=0x7fb63c00ec80 comp rx=0 tx=0).stop 2023-09-12T14:56:16.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.180+0000 7fb653433640 1 -- 172.21.15.131:0/3628681908 shutdown_connections 2023-09-12T14:56:16.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.180+0000 7fb653433640 1 --2- 172.21.15.131:0/3628681908 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fb630065b20 0x7fb630067fe0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.180+0000 7fb653433640 1 --2- 172.21.15.131:0/3628681908 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb64c158180 0x7fb64c13e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.180+0000 7fb653433640 1 --2- 172.21.15.131:0/3628681908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb64c0ab730 0x7fb64c14f4a0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.180+0000 7fb653433640 1 --2- 172.21.15.131:0/3628681908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb64c06f390 0x7fb64c14c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.180+0000 7fb653433640 1 -- 172.21.15.131:0/3628681908 >> 172.21.15.131:0/3628681908 conn(0x7fb64c069840 msgr2=0x7fb64c072320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:16.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.180+0000 7fb653433640 1 -- 172.21.15.131:0/3628681908 shutdown_connections 2023-09-12T14:56:16.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.180+0000 7fb653433640 1 -- 172.21.15.131:0/3628681908 wait complete. 2023-09-12T14:56:16.197 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:56:16.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7b08b2640 1 -- 172.21.15.131:0/2379597012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc794005e20 msgr2=0x7fc7940062a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:16.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7b08b2640 1 --2- 172.21.15.131:0/2379597012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc794005e20 0x7fc7940062a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009ee0 tx=0x7fc7a401d660 comp rx=0 tx=0).stop 2023-09-12T14:56:16.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7aad9d640 1 -- 172.21.15.131:0/2379597012 reap_dead start 2023-09-12T14:56:16.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7b08b2640 1 -- 172.21.15.131:0/2379597012 shutdown_connections 2023-09-12T14:56:16.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7b08b2640 1 -- 172.21.15.131:0/2379597012 >> 172.21.15.131:0/2379597012 conn(0x7fc7ac069840 msgr2=0x7fc7ac0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:16.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7b08b2640 1 -- 172.21.15.131:0/2379597012 shutdown_connections 2023-09-12T14:56:16.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7b08b2640 1 -- 172.21.15.131:0/2379597012 wait complete. 2023-09-12T14:56:16.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7b08b2640 1 Processor -- start 2023-09-12T14:56:16.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7b08b2640 1 -- start start 2023-09-12T14:56:16.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7b08b2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc794005e20 0x7fc7ac13b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:16.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7b08b2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7ac1e2130 0x7fc7ac1e25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7b08b2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7ac159910 0x7fc7ac159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7b08b2640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac15a2d0 con 0x7fc7ac1e2130 2023-09-12T14:56:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7b08b2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac1589f0 con 0x7fc7ac159910 2023-09-12T14:56:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7b08b2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac13b9b0 con 0x7fc794005e20 2023-09-12T14:56:16.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7a9d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7ac1e2130 0x7fc7ac1e25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:16.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7a9d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7ac1e2130 0x7fc7ac1e25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49152/0 (socket says 172.21.15.131:49152) 2023-09-12T14:56:16.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7a9d9b640 1 -- 172.21.15.131:0/369277027 learned_addr learned my addr 172.21.15.131:0/369277027 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:16.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7aad9d640 1 --2- 172.21.15.131:0/369277027 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7ac159910 0x7fc7ac159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:16.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7aa59c640 1 --2- 172.21.15.131:0/369277027 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc794005e20 0x7fc7ac13b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:16.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7a9d9b640 1 -- 172.21.15.131:0/369277027 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc794005e20 msgr2=0x7fc7ac13b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:16.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7a9d9b640 1 --2- 172.21.15.131:0/369277027 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc794005e20 0x7fc7ac13b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7a9d9b640 1 -- 172.21.15.131:0/369277027 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7ac159910 msgr2=0x7fc7ac159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:16.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7a9d9b640 1 --2- 172.21.15.131:0/369277027 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7ac159910 0x7fc7ac159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7a9d9b640 1 -- 172.21.15.131:0/369277027 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ac13bc90 con 0x7fc7ac1e2130 2023-09-12T14:56:16.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7aa59c640 1 --2- 172.21.15.131:0/369277027 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc794005e20 0x7fc7ac13b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:16.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7a9d9b640 1 --2- 172.21.15.131:0/369277027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7ac1e2130 0x7fc7ac1e25b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc7a00107e0 tx=0x7fc7a0010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:16.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc79affd640 1 -- 172.21.15.131:0/369277027 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a00095b0 con 0x7fc7ac1e2130 2023-09-12T14:56:16.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc79affd640 1 -- 172.21.15.131:0/369277027 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc7a000ece0 con 0x7fc7ac1e2130 2023-09-12T14:56:16.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7b08b2640 1 -- 172.21.15.131:0/369277027 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7ac13bf70 con 0x7fc7ac1e2130 2023-09-12T14:56:16.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc79affd640 1 -- 172.21.15.131:0/369277027 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0005280 con 0x7fc7ac1e2130 2023-09-12T14:56:16.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.320+0000 7fc7b08b2640 1 -- 172.21.15.131:0/369277027 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7ac13c530 con 0x7fc7ac1e2130 2023-09-12T14:56:16.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.324+0000 7fc7b08b2640 1 -- 172.21.15.131:0/369277027 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7ac13c6f0 con 0x7fc7ac1e2130 2023-09-12T14:56:16.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.324+0000 7fc79affd640 1 -- 172.21.15.131:0/369277027 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fc7a00053e0 con 0x7fc7ac1e2130 2023-09-12T14:56:16.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.324+0000 7fc79affd640 1 --2- 172.21.15.131:0/369277027 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc78c065b50 0x7fc78c068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:16.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.324+0000 7fc79affd640 1 -- 172.21.15.131:0/369277027 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7a0016070 con 0x7fc7ac1e2130 2023-09-12T14:56:16.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.324+0000 7fc7aa59c640 1 --2- 172.21.15.131:0/369277027 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc78c065b50 0x7fc78c068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:16.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.324+0000 7fc7aa59c640 1 --2- 172.21.15.131:0/369277027 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc78c065b50 0x7fc78c068010 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc79c000c10 tx=0x7fc79c0067b0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:16.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.328+0000 7fc79affd640 1 -- 172.21.15.131:0/369277027 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fc7a0051940 con 0x7fc7ac1e2130 2023-09-12T14:56:16.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.480+0000 7fc7b08b2640 1 -- 172.21.15.131:0/369277027 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7ac073580 con 0x7fc7ac1e2130 2023-09-12T14:56:16.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.484+0000 7fc79affd640 1 -- 172.21.15.131:0/369277027 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16605 (secure 0 0 0) 0x7fc7a005b020 con 0x7fc7ac1e2130 2023-09-12T14:56:16.488 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:56:16.488 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":33,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:55:57.898819+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:55:55.034593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-09-12T14:55:55.855927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:56:16.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.488+0000 7fc7b08b2640 1 -- 172.21.15.131:0/369277027 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc78c065b50 msgr2=0x7fc78c068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:16.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.488+0000 7fc7b08b2640 1 --2- 172.21.15.131:0/369277027 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc78c065b50 0x7fc78c068010 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc79c000c10 tx=0x7fc79c0067b0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.488+0000 7fc7b08b2640 1 -- 172.21.15.131:0/369277027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7ac1e2130 msgr2=0x7fc7ac1e25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:16.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.488+0000 7fc7b08b2640 1 --2- 172.21.15.131:0/369277027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7ac1e2130 0x7fc7ac1e25b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc7a00107e0 tx=0x7fc7a0010ca0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.488+0000 7fc7b08b2640 1 -- 172.21.15.131:0/369277027 shutdown_connections 2023-09-12T14:56:16.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.488+0000 7fc7b08b2640 1 --2- 172.21.15.131:0/369277027 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fc78c065b50 0x7fc78c068010 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.488+0000 7fc7b08b2640 1 --2- 172.21.15.131:0/369277027 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7ac159910 0x7fc7ac159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.488+0000 7fc7b08b2640 1 --2- 172.21.15.131:0/369277027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7ac1e2130 0x7fc7ac1e25b0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.488+0000 7fc7b08b2640 1 --2- 172.21.15.131:0/369277027 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc794005e20 0x7fc7ac13b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.488+0000 7fc7b08b2640 1 -- 172.21.15.131:0/369277027 >> 172.21.15.131:0/369277027 conn(0x7fc7ac069840 msgr2=0x7fc7ac0cacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:16.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.488+0000 7fc7b08b2640 1 -- 172.21.15.131:0/369277027 shutdown_connections 2023-09-12T14:56:16.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.488+0000 7fc7b08b2640 1 -- 172.21.15.131:0/369277027 wait complete. 2023-09-12T14:56:16.508 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-12T14:54:45.654004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-12T14:55:55.034593+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-09-12T14:55:55.855927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-12T14:56:16.509 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-12T14:56:16.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.632+0000 7f6ae69fe640 1 -- 172.21.15.131:0/1966055304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ae0142520 msgr2=0x7f6ae0069240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:16.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.632+0000 7f6ae69fe640 1 --2- 172.21.15.131:0/1966055304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ae0142520 0x7f6ae0069240 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00b0ac0 tx=0x7f6ad001c720 comp rx=0 tx=0).stop 2023-09-12T14:56:16.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.632+0000 7f6ae69fe640 1 -- 172.21.15.131:0/1966055304 shutdown_connections 2023-09-12T14:56:16.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.632+0000 7f6ae69fe640 1 --2- 172.21.15.131:0/1966055304 >> v2:172.21.15.187:3300/0 conn(0x7f6ae0159260 0x7f6ae0069780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.632+0000 7f6ae69fe640 1 --2- 172.21.15.131:0/1966055304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ae0142520 0x7f6ae0069240 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.632+0000 7f6ae69fe640 1 -- 172.21.15.131:0/1966055304 >> 172.21.15.131:0/1966055304 conn(0x7f6ae006ab10 msgr2=0x7f6ae00af730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:16.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.632+0000 7f6ae69fe640 1 -- 172.21.15.131:0/1966055304 shutdown_connections 2023-09-12T14:56:16.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.632+0000 7f6ae69fe640 1 -- 172.21.15.131:0/1966055304 wait complete. 2023-09-12T14:56:16.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.636+0000 7f6ae69fe640 1 Processor -- start 2023-09-12T14:56:16.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.636+0000 7f6ae69fe640 1 -- start start 2023-09-12T14:56:16.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.636+0000 7f6ae69fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ae006da30 0x7f6ae0129290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:16.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.636+0000 7f6ae69fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ae0142520 0x7f6ae0153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:16.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.636+0000 7f6ae69fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ae0159260 0x7f6ae015a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:16.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.636+0000 7f6ae69fe640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae0136fd0 con 0x7f6ae006da30 2023-09-12T14:56:16.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.636+0000 7f6ae69fe640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae015d0c0 con 0x7f6ae0159260 2023-09-12T14:56:16.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.636+0000 7f6ae69fe640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae015c4f0 con 0x7f6ae0142520 2023-09-12T14:56:16.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.636+0000 7f6adffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ae006da30 0x7f6ae0129290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:16.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.636+0000 7f6adffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ae006da30 0x7f6ae0129290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49180/0 (socket says 172.21.15.131:49180) 2023-09-12T14:56:16.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.636+0000 7f6adffff640 1 -- 172.21.15.131:0/1977753749 learned_addr learned my addr 172.21.15.131:0/1977753749 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:16.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.636+0000 7f6adffff640 1 -- 172.21.15.131:0/1977753749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ae0142520 msgr2=0x7f6ae0153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:16.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.636+0000 7f6adffff640 1 --2- 172.21.15.131:0/1977753749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ae0142520 0x7f6ae0153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.636+0000 7f6adffff640 1 -- 172.21.15.131:0/1977753749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ae0159260 msgr2=0x7f6ae015a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:16.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.636+0000 7f6ae4f9b640 1 --2- 172.21.15.131:0/1977753749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ae0159260 0x7f6ae015a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:16.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.636+0000 7f6adffff640 1 --2- 172.21.15.131:0/1977753749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ae0159260 0x7f6ae015a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.636+0000 7f6adffff640 1 -- 172.21.15.131:0/1977753749 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad00094a0 con 0x7f6ae006da30 2023-09-12T14:56:16.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.636+0000 7f6ae4f9b640 1 --2- 172.21.15.131:0/1977753749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ae0159260 0x7f6ae015a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:16.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.636+0000 7f6adffff640 1 --2- 172.21.15.131:0/1977753749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ae006da30 0x7f6ae0129290 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0000c10 tx=0x7f6ad00a5750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:16.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.636+0000 7f6adcff9640 1 -- 172.21.15.131:0/1977753749 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad00042f0 con 0x7f6ae006da30 2023-09-12T14:56:16.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.636+0000 7f6adcff9640 1 -- 172.21.15.131:0/1977753749 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6ad0003600 con 0x7f6ae006da30 2023-09-12T14:56:16.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.636+0000 7f6ae69fe640 1 -- 172.21.15.131:0/1977753749 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae00cd650 con 0x7f6ae006da30 2023-09-12T14:56:16.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.636+0000 7f6adcff9640 1 -- 172.21.15.131:0/1977753749 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad00039b0 con 0x7f6ae006da30 2023-09-12T14:56:16.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.636+0000 7f6ae69fe640 1 -- 172.21.15.131:0/1977753749 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae015d550 con 0x7f6ae006da30 2023-09-12T14:56:16.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.636+0000 7f6ae69fe640 1 -- 172.21.15.131:0/1977753749 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ae0073580 con 0x7f6ae006da30 2023-09-12T14:56:16.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.636+0000 7f6adcff9640 1 -- 172.21.15.131:0/1977753749 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f6ad00b4040 con 0x7f6ae006da30 2023-09-12T14:56:16.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.640+0000 7f6adcff9640 1 --2- 172.21.15.131:0/1977753749 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6acc065b50 0x7f6acc068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:16.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.640+0000 7f6adcff9640 1 -- 172.21.15.131:0/1977753749 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ad011cfd0 con 0x7f6ae006da30 2023-09-12T14:56:16.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.640+0000 7f6adf7fe640 1 --2- 172.21.15.131:0/1977753749 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6acc065b50 0x7f6acc068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:16.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.640+0000 7f6adf7fe640 1 --2- 172.21.15.131:0/1977753749 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6acc065b50 0x7f6acc068010 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6ae01f5430 tx=0x7f6ad400b370 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:16.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.644+0000 7f6adcff9640 1 -- 172.21.15.131:0/1977753749 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f6ad00ea890 con 0x7f6ae006da30 2023-09-12T14:56:16.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.796+0000 7f6ae69fe640 1 -- 172.21.15.131:0/1977753749 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6ae013a290 con 0x7f6ae006da30 2023-09-12T14:56:16.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.796+0000 7f6adcff9640 1 -- 172.21.15.131:0/1977753749 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6ad00b2020 con 0x7f6ae006da30 2023-09-12T14:56:16.799 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-09-12T14:56:16.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.796+0000 7f6ae69fe640 1 -- 172.21.15.131:0/1977753749 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6acc065b50 msgr2=0x7f6acc068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:16.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.796+0000 7f6ae69fe640 1 --2- 172.21.15.131:0/1977753749 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6acc065b50 0x7f6acc068010 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6ae01f5430 tx=0x7f6ad400b370 comp rx=0 tx=0).stop 2023-09-12T14:56:16.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.796+0000 7f6ae69fe640 1 -- 172.21.15.131:0/1977753749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ae006da30 msgr2=0x7f6ae0129290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:16.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.796+0000 7f6ae69fe640 1 --2- 172.21.15.131:0/1977753749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ae006da30 0x7f6ae0129290 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0000c10 tx=0x7f6ad00a5750 comp rx=0 tx=0).stop 2023-09-12T14:56:16.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.796+0000 7f6ae69fe640 1 -- 172.21.15.131:0/1977753749 shutdown_connections 2023-09-12T14:56:16.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.796+0000 7f6ae69fe640 1 --2- 172.21.15.131:0/1977753749 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f6acc065b50 0x7f6acc068010 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.796+0000 7f6ae69fe640 1 --2- 172.21.15.131:0/1977753749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ae0142520 0x7f6ae0153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.796+0000 7f6ae69fe640 1 --2- 172.21.15.131:0/1977753749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ae0159260 0x7f6ae015a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.796+0000 7f6ae69fe640 1 --2- 172.21.15.131:0/1977753749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ae006da30 0x7f6ae0129290 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.796+0000 7f6ae69fe640 1 -- 172.21.15.131:0/1977753749 >> 172.21.15.131:0/1977753749 conn(0x7f6ae006ab10 msgr2=0x7f6ae013a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:16.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.796+0000 7f6ae69fe640 1 -- 172.21.15.131:0/1977753749 shutdown_connections 2023-09-12T14:56:16.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.796+0000 7f6ae69fe640 1 -- 172.21.15.131:0/1977753749 wait complete. 2023-09-12T14:56:16.818 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-12T14:56:16.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd459c9640 1 -- 172.21.15.131:0/20092983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd400d1680 msgr2=0x7efd400b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:16.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd459c9640 1 --2- 172.21.15.131:0/20092983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd400d1680 0x7efd400b0750 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7efd28009840 tx=0x7efd2801c7d0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd459c9640 1 -- 172.21.15.131:0/20092983 shutdown_connections 2023-09-12T14:56:16.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd459c9640 1 --2- 172.21.15.131:0/20092983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd4015e9f0 0x7efd400b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd459c9640 1 --2- 172.21.15.131:0/20092983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd400d1680 0x7efd400b0750 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd459c9640 1 -- 172.21.15.131:0/20092983 >> 172.21.15.131:0/20092983 conn(0x7efd4009a200 msgr2=0x7efd400d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:16.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd459c9640 1 -- 172.21.15.131:0/20092983 shutdown_connections 2023-09-12T14:56:16.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd459c9640 1 -- 172.21.15.131:0/20092983 wait complete. 2023-09-12T14:56:16.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd459c9640 1 Processor -- start 2023-09-12T14:56:16.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd459c9640 1 -- start start 2023-09-12T14:56:16.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd459c9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd400cf2d0 0x7efd400ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:16.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd459c9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd400d1680 0x7efd4014b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:16.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd459c9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd4015e9f0 0x7efd4014ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:16.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd459c9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd400c8a50 con 0x7efd400cf2d0 2023-09-12T14:56:16.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd459c9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd40154e60 con 0x7efd4015e9f0 2023-09-12T14:56:16.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd459c9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd400c8ca0 con 0x7efd400d1680 2023-09-12T14:56:16.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd3effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd400cf2d0 0x7efd400ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:16.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd3effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd400cf2d0 0x7efd400ca6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49194/0 (socket says 172.21.15.131:49194) 2023-09-12T14:56:16.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd3effd640 1 -- 172.21.15.131:0/772970402 learned_addr learned my addr 172.21.15.131:0/772970402 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:16.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd3effd640 1 -- 172.21.15.131:0/772970402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd400d1680 msgr2=0x7efd4014b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:16.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd3effd640 1 --2- 172.21.15.131:0/772970402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd400d1680 0x7efd4014b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd3effd640 1 -- 172.21.15.131:0/772970402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd4015e9f0 msgr2=0x7efd4014ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:16.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd3effd640 1 --2- 172.21.15.131:0/772970402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd4015e9f0 0x7efd4014ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:16.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd3effd640 1 -- 172.21.15.131:0/772970402 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd280094a0 con 0x7efd400cf2d0 2023-09-12T14:56:16.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd3effd640 1 --2- 172.21.15.131:0/772970402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd400cf2d0 0x7efd400ca6a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7efd3000c460 tx=0x7efd3000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:16.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd3cff9640 1 -- 172.21.15.131:0/772970402 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd30004230 con 0x7efd400cf2d0 2023-09-12T14:56:16.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd3cff9640 1 -- 172.21.15.131:0/772970402 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efd300044a0 con 0x7efd400cf2d0 2023-09-12T14:56:16.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd459c9640 1 -- 172.21.15.131:0/772970402 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd401550b0 con 0x7efd400cf2d0 2023-09-12T14:56:16.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd3cff9640 1 -- 172.21.15.131:0/772970402 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd30012890 con 0x7efd400cf2d0 2023-09-12T14:56:16.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd459c9640 1 -- 172.21.15.131:0/772970402 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd40155670 con 0x7efd400cf2d0 2023-09-12T14:56:16.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.944+0000 7efd459c9640 1 -- 172.21.15.131:0/772970402 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd40155860 con 0x7efd400cf2d0 2023-09-12T14:56:16.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.948+0000 7efd3cff9640 1 -- 172.21.15.131:0/772970402 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7efd30005020 con 0x7efd400cf2d0 2023-09-12T14:56:16.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.948+0000 7efd3cff9640 1 --2- 172.21.15.131:0/772970402 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7efd20065b20 0x7efd20067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:16.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.948+0000 7efd367fc640 1 --2- 172.21.15.131:0/772970402 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7efd20065b20 0x7efd20067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:16.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.948+0000 7efd3cff9640 1 -- 172.21.15.131:0/772970402 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd30083ae0 con 0x7efd400cf2d0 2023-09-12T14:56:16.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.948+0000 7efd367fc640 1 --2- 172.21.15.131:0/772970402 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7efd20065b20 0x7efd20067fe0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efd280af000 tx=0x7efd28019570 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:16.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:16.952+0000 7efd3cff9640 1 -- 172.21.15.131:0/772970402 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7efd30055060 con 0x7efd400cf2d0 2023-09-12T14:56:17.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.104+0000 7efd459c9640 1 -- 172.21.15.131:0/772970402 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efd400afee0 con 0x7efd400cf2d0 2023-09-12T14:56:17.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.104+0000 7efd3cff9640 1 -- 172.21.15.131:0/772970402 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7efd30054bf0 con 0x7efd400cf2d0 2023-09-12T14:56:17.107 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-09-12T14:56:17.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.104+0000 7efd459c9640 1 -- 172.21.15.131:0/772970402 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7efd20065b20 msgr2=0x7efd20067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:17.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.104+0000 7efd459c9640 1 --2- 172.21.15.131:0/772970402 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7efd20065b20 0x7efd20067fe0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efd280af000 tx=0x7efd28019570 comp rx=0 tx=0).stop 2023-09-12T14:56:17.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.104+0000 7efd459c9640 1 -- 172.21.15.131:0/772970402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd400cf2d0 msgr2=0x7efd400ca6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:17.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.104+0000 7efd459c9640 1 --2- 172.21.15.131:0/772970402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd400cf2d0 0x7efd400ca6a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7efd3000c460 tx=0x7efd3000c920 comp rx=0 tx=0).stop 2023-09-12T14:56:17.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.104+0000 7efd459c9640 1 -- 172.21.15.131:0/772970402 shutdown_connections 2023-09-12T14:56:17.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.104+0000 7efd459c9640 1 --2- 172.21.15.131:0/772970402 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7efd20065b20 0x7efd20067fe0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:17.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.104+0000 7efd459c9640 1 --2- 172.21.15.131:0/772970402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd400d1680 0x7efd4014b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:17.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.104+0000 7efd459c9640 1 --2- 172.21.15.131:0/772970402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd4015e9f0 0x7efd4014ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:17.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.104+0000 7efd459c9640 1 --2- 172.21.15.131:0/772970402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd400cf2d0 0x7efd400ca6a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:17.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.104+0000 7efd459c9640 1 -- 172.21.15.131:0/772970402 >> 172.21.15.131:0/772970402 conn(0x7efd4009a200 msgr2=0x7efd400b2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:17.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.104+0000 7efd459c9640 1 -- 172.21.15.131:0/772970402 shutdown_connections 2023-09-12T14:56:17.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.104+0000 7efd459c9640 1 -- 172.21.15.131:0/772970402 wait complete. 2023-09-12T14:56:17.125 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-12T14:56:17.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.248+0000 7f12c17ac640 1 -- 172.21.15.131:0/2300907877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12bc0b1760 msgr2=0x7f12bc15e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:17.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.248+0000 7f12c17ac640 1 --2- 172.21.15.131:0/2300907877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12bc0b1760 0x7f12bc15e820 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009cb0 tx=0x7f12b401cd80 comp rx=0 tx=0).stop 2023-09-12T14:56:17.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.248+0000 7f12c17ac640 1 -- 172.21.15.131:0/2300907877 shutdown_connections 2023-09-12T14:56:17.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.248+0000 7f12c17ac640 1 --2- 172.21.15.131:0/2300907877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12bc0b1760 0x7f12bc15e820 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:17.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.248+0000 7f12c17ac640 1 --2- 172.21.15.131:0/2300907877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12bc0cf2d0 0x7f12bc15e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:17.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.248+0000 7f12c17ac640 1 -- 172.21.15.131:0/2300907877 >> 172.21.15.131:0/2300907877 conn(0x7f12bc09a200 msgr2=0x7f12bc0d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:17.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.248+0000 7f12c17ac640 1 -- 172.21.15.131:0/2300907877 shutdown_connections 2023-09-12T14:56:17.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.248+0000 7f12c17ac640 1 -- 172.21.15.131:0/2300907877 wait complete. 2023-09-12T14:56:17.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.248+0000 7f12c17ac640 1 Processor -- start 2023-09-12T14:56:17.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.248+0000 7f12c17ac640 1 -- start start 2023-09-12T14:56:17.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12c17ac640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12bc0b1760 0x7f12bc15b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:17.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12c17ac640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12bc0cf2d0 0x7f12bc152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:17.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12c17ac640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12bc0d1680 0x7f12bc15c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:17.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12c17ac640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12bc15b5d0 con 0x7f12bc0cf2d0 2023-09-12T14:56:17.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12c17ac640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12bc0b3ed0 con 0x7f12bc0d1680 2023-09-12T14:56:17.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12c17ac640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12bc15b8b0 con 0x7f12bc0b1760 2023-09-12T14:56:17.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12ba7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12bc0cf2d0 0x7f12bc152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:17.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12ba7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12bc0cf2d0 0x7f12bc152480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49212/0 (socket says 172.21.15.131:49212) 2023-09-12T14:56:17.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12ba7fc640 1 -- 172.21.15.131:0/494918196 learned_addr learned my addr 172.21.15.131:0/494918196 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:17.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12baffd640 1 --2- 172.21.15.131:0/494918196 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12bc0b1760 0x7f12bc15b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:17.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12bb7fe640 1 --2- 172.21.15.131:0/494918196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12bc0d1680 0x7f12bc15c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:17.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12ba7fc640 1 -- 172.21.15.131:0/494918196 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12bc0b1760 msgr2=0x7f12bc15b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:17.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12ba7fc640 1 --2- 172.21.15.131:0/494918196 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12bc0b1760 0x7f12bc15b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:17.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12ba7fc640 1 -- 172.21.15.131:0/494918196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12bc0d1680 msgr2=0x7f12bc15c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:17.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12ba7fc640 1 --2- 172.21.15.131:0/494918196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12bc0d1680 0x7f12bc15c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:17.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12ba7fc640 1 -- 172.21.15.131:0/494918196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b40098e0 con 0x7f12bc0cf2d0 2023-09-12T14:56:17.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12ba7fc640 1 --2- 172.21.15.131:0/494918196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12bc0cf2d0 0x7f12bc152480 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f12b000c390 tx=0x7f12b000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:17.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12a37fe640 1 -- 172.21.15.131:0/494918196 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b00052d0 con 0x7f12bc0cf2d0 2023-09-12T14:56:17.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12a37fe640 1 -- 172.21.15.131:0/494918196 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f12b0003600 con 0x7f12bc0cf2d0 2023-09-12T14:56:17.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12c17ac640 1 -- 172.21.15.131:0/494918196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12bc15bb90 con 0x7f12bc0cf2d0 2023-09-12T14:56:17.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12a37fe640 1 -- 172.21.15.131:0/494918196 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b00039f0 con 0x7f12bc0cf2d0 2023-09-12T14:56:17.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12c17ac640 1 -- 172.21.15.131:0/494918196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12bc1529c0 con 0x7f12bc0cf2d0 2023-09-12T14:56:17.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12c17ac640 1 -- 172.21.15.131:0/494918196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12bc073580 con 0x7f12bc0cf2d0 2023-09-12T14:56:17.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12a37fe640 1 -- 172.21.15.131:0/494918196 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f12b0003b50 con 0x7f12bc0cf2d0 2023-09-12T14:56:17.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12a37fe640 1 --2- 172.21.15.131:0/494918196 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f12a8065b20 0x7f12a8067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:17.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12baffd640 1 --2- 172.21.15.131:0/494918196 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f12a8065b20 0x7f12a8067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:17.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12a37fe640 1 -- 172.21.15.131:0/494918196 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12b0082ce0 con 0x7f12bc0cf2d0 2023-09-12T14:56:17.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.252+0000 7f12baffd640 1 --2- 172.21.15.131:0/494918196 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f12a8065b20 0x7f12a8067fe0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0058d0 tx=0x7f12ac005860 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:17.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.256+0000 7f12a37fe640 1 -- 172.21.15.131:0/494918196 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f12b00506c0 con 0x7f12bc0cf2d0 2023-09-12T14:56:17.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.400+0000 7f12c17ac640 1 -- 172.21.15.131:0/494918196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f12bc0b6710 con 0x7f12bc0cf2d0 2023-09-12T14:56:17.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.400+0000 7f12a37fe640 1 -- 172.21.15.131:0/494918196 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7f12b0053ce0 con 0x7f12bc0cf2d0 2023-09-12T14:56:17.402 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-09-12T14:56:17.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.400+0000 7f12c17ac640 1 -- 172.21.15.131:0/494918196 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f12a8065b20 msgr2=0x7f12a8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:17.405 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.400+0000 7f12c17ac640 1 --2- 172.21.15.131:0/494918196 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f12a8065b20 0x7f12a8067fe0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0058d0 tx=0x7f12ac005860 comp rx=0 tx=0).stop 2023-09-12T14:56:17.405 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.400+0000 7f12c17ac640 1 -- 172.21.15.131:0/494918196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12bc0cf2d0 msgr2=0x7f12bc152480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:17.405 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.400+0000 7f12c17ac640 1 --2- 172.21.15.131:0/494918196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12bc0cf2d0 0x7f12bc152480 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f12b000c390 tx=0x7f12b000c850 comp rx=0 tx=0).stop 2023-09-12T14:56:17.405 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.400+0000 7f12c17ac640 1 -- 172.21.15.131:0/494918196 shutdown_connections 2023-09-12T14:56:17.405 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.400+0000 7f12c17ac640 1 --2- 172.21.15.131:0/494918196 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f12a8065b20 0x7f12a8067fe0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:17.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.400+0000 7f12c17ac640 1 --2- 172.21.15.131:0/494918196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12bc0d1680 0x7f12bc15c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:17.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.400+0000 7f12c17ac640 1 --2- 172.21.15.131:0/494918196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12bc0cf2d0 0x7f12bc152480 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:17.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.400+0000 7f12c17ac640 1 --2- 172.21.15.131:0/494918196 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12bc0b1760 0x7f12bc15b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:17.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.400+0000 7f12c17ac640 1 -- 172.21.15.131:0/494918196 >> 172.21.15.131:0/494918196 conn(0x7f12bc09a200 msgr2=0x7f12bc06abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:17.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.400+0000 7f12c17ac640 1 -- 172.21.15.131:0/494918196 shutdown_connections 2023-09-12T14:56:17.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.400+0000 7f12c17ac640 1 -- 172.21.15.131:0/494918196 wait complete. 2023-09-12T14:56:17.420 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T14:56:17.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a33d4640 1 -- 172.21.15.131:0/1225579780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14880052b0 msgr2=0x7f148800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:17.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a33d4640 1 --2- 172.21.15.131:0/1225579780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14880052b0 0x7f148800b990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f14980016b0 tx=0x7f149801c7a0 comp rx=0 tx=0).stop 2023-09-12T14:56:17.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a1971640 1 -- 172.21.15.131:0/1225579780 reap_dead start 2023-09-12T14:56:17.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a33d4640 1 -- 172.21.15.131:0/1225579780 shutdown_connections 2023-09-12T14:56:17.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a33d4640 1 -- 172.21.15.131:0/1225579780 >> 172.21.15.131:0/1225579780 conn(0x7f149c06b3c0 msgr2=0x7f149c06b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:17.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a33d4640 1 -- 172.21.15.131:0/1225579780 shutdown_connections 2023-09-12T14:56:17.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a33d4640 1 -- 172.21.15.131:0/1225579780 wait complete. 2023-09-12T14:56:17.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a33d4640 1 Processor -- start 2023-09-12T14:56:17.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a33d4640 1 -- start start 2023-09-12T14:56:17.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a33d4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14880052b0 0x7f149c141bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:17.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a33d4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f149c15c720 0x7f149c15cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:17.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a33d4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f149c06b7d0 0x7f149c06bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:17.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a33d4640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c06c1c0 con 0x7f149c15c720 2023-09-12T14:56:17.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a33d4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c15d0e0 con 0x7f149c06b7d0 2023-09-12T14:56:17.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a33d4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f149c0c9ab0 con 0x7f14880052b0 2023-09-12T14:56:17.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a096f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f149c15c720 0x7f149c15cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:17.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a096f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f149c15c720 0x7f149c15cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49224/0 (socket says 172.21.15.131:49224) 2023-09-12T14:56:17.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a096f640 1 -- 172.21.15.131:0/447832179 learned_addr learned my addr 172.21.15.131:0/447832179 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:17.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a1170640 1 --2- 172.21.15.131:0/447832179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14880052b0 0x7f149c141bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:17.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a1971640 1 --2- 172.21.15.131:0/447832179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f149c06b7d0 0x7f149c06bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:17.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a096f640 1 -- 172.21.15.131:0/447832179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14880052b0 msgr2=0x7f149c141bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:17.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a096f640 1 --2- 172.21.15.131:0/447832179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14880052b0 0x7f149c141bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:17.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a096f640 1 -- 172.21.15.131:0/447832179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f149c06b7d0 msgr2=0x7f149c06bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:17.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a096f640 1 --2- 172.21.15.131:0/447832179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f149c06b7d0 0x7f149c06bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:17.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a096f640 1 -- 172.21.15.131:0/447832179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149c1420f0 con 0x7f149c15c720 2023-09-12T14:56:17.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a096f640 1 --2- 172.21.15.131:0/447832179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f149c15c720 0x7f149c15cba0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f149400f7a0 tx=0x7f149400fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:17.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f148dffb640 1 -- 172.21.15.131:0/447832179 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149400d5e0 con 0x7f149c15c720 2023-09-12T14:56:17.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f148dffb640 1 -- 172.21.15.131:0/447832179 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f149400d850 con 0x7f149c15c720 2023-09-12T14:56:17.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a33d4640 1 -- 172.21.15.131:0/447832179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f149c15d2e0 con 0x7f149c15c720 2023-09-12T14:56:17.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f148dffb640 1 -- 172.21.15.131:0/447832179 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149400dc60 con 0x7f149c15c720 2023-09-12T14:56:17.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a33d4640 1 -- 172.21.15.131:0/447832179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f149c15d8a0 con 0x7f149c15c720 2023-09-12T14:56:17.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.536+0000 7f14a33d4640 1 -- 172.21.15.131:0/447832179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f149c073580 con 0x7f149c15c720 2023-09-12T14:56:17.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.540+0000 7f148dffb640 1 -- 172.21.15.131:0/447832179 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f1494008070 con 0x7f149c15c720 2023-09-12T14:56:17.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.540+0000 7f148dffb640 1 --2- 172.21.15.131:0/447832179 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1480065b50 0x7f1480068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:17.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.540+0000 7f148dffb640 1 -- 172.21.15.131:0/447832179 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1494085ce0 con 0x7f149c15c720 2023-09-12T14:56:17.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.540+0000 7f14a1170640 1 --2- 172.21.15.131:0/447832179 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1480065b50 0x7f1480068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:17.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.540+0000 7f14a1170640 1 --2- 172.21.15.131:0/447832179 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1480065b50 0x7f1480068010 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f149c06c3e0 tx=0x7f1490005c80 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:17.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.544+0000 7f148dffb640 1 -- 172.21.15.131:0/447832179 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f14940536c0 con 0x7f149c15c720 2023-09-12T14:56:17.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.716+0000 7f14a33d4640 1 -- 172.21.15.131:0/447832179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f149c0b1a80 con 0x7f149c15c720 2023-09-12T14:56:17.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.716+0000 7f148dffb640 1 -- 172.21.15.131:0/447832179 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f149401a090 con 0x7f149c15c720 2023-09-12T14:56:17.721 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:56:17.721 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4676,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/3890238785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3890238785},{"type":"v1","addr":"172.21.15.187:6834","nonce":3890238785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4978,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1436257801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":1436257801},{"type":"v1","addr":"172.21.15.187:6835","nonce":1436257801}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4990,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/806466310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":806466310},{"type":"v1","addr":"172.21.15.131:6837","nonce":806466310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-12T14:55:56.870775+0000","modified":"2023-09-12T14:55:58.869201+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/871788191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":871788191},{"type":"v1","addr":"172.21.15.131:6835","nonce":871788191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-09-12T14:56:17.721 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 13 2023-09-12T14:56:17.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.720+0000 7f14a33d4640 1 -- 172.21.15.131:0/447832179 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1480065b50 msgr2=0x7f1480068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:17.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.720+0000 7f14a33d4640 1 --2- 172.21.15.131:0/447832179 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1480065b50 0x7f1480068010 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f149c06c3e0 tx=0x7f1490005c80 comp rx=0 tx=0).stop 2023-09-12T14:56:17.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.720+0000 7f14a33d4640 1 -- 172.21.15.131:0/447832179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f149c15c720 msgr2=0x7f149c15cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:17.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.720+0000 7f14a33d4640 1 --2- 172.21.15.131:0/447832179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f149c15c720 0x7f149c15cba0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f149400f7a0 tx=0x7f149400fc60 comp rx=0 tx=0).stop 2023-09-12T14:56:17.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.720+0000 7f14a33d4640 1 -- 172.21.15.131:0/447832179 shutdown_connections 2023-09-12T14:56:17.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.720+0000 7f14a33d4640 1 --2- 172.21.15.131:0/447832179 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1480065b50 0x7f1480068010 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:17.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.720+0000 7f14a33d4640 1 --2- 172.21.15.131:0/447832179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f149c06b7d0 0x7f149c06bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:17.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.720+0000 7f14a33d4640 1 --2- 172.21.15.131:0/447832179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f149c15c720 0x7f149c15cba0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:17.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.720+0000 7f14a33d4640 1 --2- 172.21.15.131:0/447832179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14880052b0 0x7f149c141bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:17.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.720+0000 7f14a33d4640 1 -- 172.21.15.131:0/447832179 >> 172.21.15.131:0/447832179 conn(0x7f149c06b3c0 msgr2=0x7f149c0b0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:17.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.720+0000 7f14a33d4640 1 -- 172.21.15.131:0/447832179 shutdown_connections 2023-09-12T14:56:17.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.720+0000 7f14a33d4640 1 -- 172.21.15.131:0/447832179 wait complete. 2023-09-12T14:56:17.738 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:56:17.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.852+0000 7f5b63bf1640 1 -- 172.21.15.131:0/2337186406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b5c0abb40 msgr2=0x7f5b5c06b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:17.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.852+0000 7f5b63bf1640 1 --2- 172.21.15.131:0/2337186406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b5c0abb40 0x7f5b5c06b4a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c0cd200 tx=0x7f5b4c01c7b0 comp rx=0 tx=0).stop 2023-09-12T14:56:17.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.852+0000 7f5b63bf1640 1 -- 172.21.15.131:0/2337186406 shutdown_connections 2023-09-12T14:56:17.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.852+0000 7f5b63bf1640 1 --2- 172.21.15.131:0/2337186406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b5c0abb40 0x7f5b5c06b4a0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:17.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.852+0000 7f5b63bf1640 1 --2- 172.21.15.131:0/2337186406 >> v2:172.21.15.187:3300/0 conn(0x7f5b5c156930 0x7f5b5c159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:17.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.852+0000 7f5b63bf1640 1 -- 172.21.15.131:0/2337186406 >> 172.21.15.131:0/2337186406 conn(0x7f5b5c069840 msgr2=0x7f5b5c0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:17.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.852+0000 7f5b63bf1640 1 -- 172.21.15.131:0/2337186406 shutdown_connections 2023-09-12T14:56:17.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.852+0000 7f5b63bf1640 1 -- 172.21.15.131:0/2337186406 wait complete. 2023-09-12T14:56:17.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.856+0000 7f5b63bf1640 1 Processor -- start 2023-09-12T14:56:17.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.856+0000 7f5b63bf1640 1 -- start start 2023-09-12T14:56:17.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.856+0000 7f5b63bf1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b5c0abb40 0x7f5b5c14ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:17.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.856+0000 7f5b63bf1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b5c0c9370 0x7f5b5c132e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:17.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.856+0000 7f5b63bf1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b5c156930 0x7f5b5c133380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:17.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.856+0000 7f5b63bf1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c1560a0 con 0x7f5b5c0abb40 2023-09-12T14:56:17.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.856+0000 7f5b63bf1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c14bf70 con 0x7f5b5c0c9370 2023-09-12T14:56:17.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.856+0000 7f5b63bf1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c14c250 con 0x7f5b5c156930 2023-09-12T14:56:17.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.856+0000 7f5b6198d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b5c0abb40 0x7f5b5c14ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:17.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.856+0000 7f5b6118c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b5c0c9370 0x7f5b5c132e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:17.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.856+0000 7f5b6198d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b5c0abb40 0x7f5b5c14ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49244/0 (socket says 172.21.15.131:49244) 2023-09-12T14:56:17.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.856+0000 7f5b6198d640 1 -- 172.21.15.131:0/3773144302 learned_addr learned my addr 172.21.15.131:0/3773144302 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:17.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.856+0000 7f5b6218e640 1 --2- 172.21.15.131:0/3773144302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b5c156930 0x7f5b5c133380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:17.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.856+0000 7f5b6198d640 1 -- 172.21.15.131:0/3773144302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b5c156930 msgr2=0x7f5b5c133380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:17.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.856+0000 7f5b6198d640 1 --2- 172.21.15.131:0/3773144302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b5c156930 0x7f5b5c133380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:17.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.856+0000 7f5b6198d640 1 -- 172.21.15.131:0/3773144302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b5c0c9370 msgr2=0x7f5b5c132e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:17.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.856+0000 7f5b6198d640 1 --2- 172.21.15.131:0/3773144302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b5c0c9370 0x7f5b5c132e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:17.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.856+0000 7f5b6198d640 1 -- 172.21.15.131:0/3773144302 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c0094a0 con 0x7f5b5c0abb40 2023-09-12T14:56:17.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.856+0000 7f5b6198d640 1 --2- 172.21.15.131:0/3773144302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b5c0abb40 0x7f5b5c14ebc0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800c9f0 tx=0x7f5b5800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:17.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.856+0000 7f5b527fc640 1 -- 172.21.15.131:0/3773144302 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b58007bb0 con 0x7f5b5c0abb40 2023-09-12T14:56:17.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.856+0000 7f5b527fc640 1 -- 172.21.15.131:0/3773144302 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5b58007e00 con 0x7f5b5c0abb40 2023-09-12T14:56:17.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.856+0000 7f5b63bf1640 1 -- 172.21.15.131:0/3773144302 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b5c14e6a0 con 0x7f5b5c0abb40 2023-09-12T14:56:17.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.856+0000 7f5b527fc640 1 -- 172.21.15.131:0/3773144302 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5800f6b0 con 0x7f5b5c0abb40 2023-09-12T14:56:17.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.856+0000 7f5b63bf1640 1 -- 172.21.15.131:0/3773144302 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b5c148d40 con 0x7f5b5c0abb40 2023-09-12T14:56:17.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.856+0000 7f5b63bf1640 1 -- 172.21.15.131:0/3773144302 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b5c073580 con 0x7f5b5c0abb40 2023-09-12T14:56:17.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.860+0000 7f5b527fc640 1 -- 172.21.15.131:0/3773144302 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f5b5800f810 con 0x7f5b5c0abb40 2023-09-12T14:56:17.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.860+0000 7f5b527fc640 1 --2- 172.21.15.131:0/3773144302 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5b40065b20 0x7f5b40067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:17.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.860+0000 7f5b527fc640 1 -- 172.21.15.131:0/3773144302 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b580839c0 con 0x7f5b5c0abb40 2023-09-12T14:56:17.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.860+0000 7f5b6118c640 1 --2- 172.21.15.131:0/3773144302 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5b40065b20 0x7f5b40067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:17.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.860+0000 7f5b6118c640 1 --2- 172.21.15.131:0/3773144302 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5b40065b20 0x7f5b40067fe0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c0af000 tx=0x7f5b4c01aff0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:17.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:17.864+0000 7f5b527fc640 1 -- 172.21.15.131:0/3773144302 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f5b58051450 con 0x7f5b5c0abb40 2023-09-12T14:56:18.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.008+0000 7f5b63bf1640 1 -- 172.21.15.131:0/3773144302 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b5c0cd4d0 con 0x7f5b5c0abb40 2023-09-12T14:56:18.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.008+0000 7f5b527fc640 1 -- 172.21.15.131:0/3773144302 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f5b58054a70 con 0x7f5b5c0abb40 2023-09-12T14:56:18.013 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:56:18.013 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":33,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:55:57.898819+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:55:55.034593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-09-12T14:55:55.855927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:56:18.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.012+0000 7f5b63bf1640 1 -- 172.21.15.131:0/3773144302 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5b40065b20 msgr2=0x7f5b40067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:18.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.012+0000 7f5b63bf1640 1 --2- 172.21.15.131:0/3773144302 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5b40065b20 0x7f5b40067fe0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c0af000 tx=0x7f5b4c01aff0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.012+0000 7f5b63bf1640 1 -- 172.21.15.131:0/3773144302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b5c0abb40 msgr2=0x7f5b5c14ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:18.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.012+0000 7f5b63bf1640 1 --2- 172.21.15.131:0/3773144302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b5c0abb40 0x7f5b5c14ebc0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800c9f0 tx=0x7f5b5800ceb0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.012+0000 7f5b63bf1640 1 -- 172.21.15.131:0/3773144302 shutdown_connections 2023-09-12T14:56:18.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.012+0000 7f5b63bf1640 1 --2- 172.21.15.131:0/3773144302 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5b40065b20 0x7f5b40067fe0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.012+0000 7f5b63bf1640 1 --2- 172.21.15.131:0/3773144302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b5c156930 0x7f5b5c133380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.012+0000 7f5b63bf1640 1 --2- 172.21.15.131:0/3773144302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b5c0c9370 0x7f5b5c132e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.012+0000 7f5b63bf1640 1 --2- 172.21.15.131:0/3773144302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b5c0abb40 0x7f5b5c14ebc0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.012+0000 7f5b63bf1640 1 -- 172.21.15.131:0/3773144302 >> 172.21.15.131:0/3773144302 conn(0x7f5b5c069840 msgr2=0x7f5b5c15a540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:18.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.012+0000 7f5b63bf1640 1 -- 172.21.15.131:0/3773144302 shutdown_connections 2023-09-12T14:56:18.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.012+0000 7f5b63bf1640 1 -- 172.21.15.131:0/3773144302 wait complete. 2023-09-12T14:56:18.030 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-12T14:56:18.031 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T14:56:18.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.140+0000 7f04a8479640 1 -- 172.21.15.131:0/2688223741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04a00c9370 msgr2=0x7f04a00c97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:18.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.140+0000 7f04a8479640 1 --2- 172.21.15.131:0/2688223741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04a00c9370 0x7f04a00c97a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0498009840 tx=0x7f049801c7d0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.140+0000 7f04a8479640 1 -- 172.21.15.131:0/2688223741 shutdown_connections 2023-09-12T14:56:18.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.140+0000 7f04a8479640 1 --2- 172.21.15.131:0/2688223741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04a00c9370 0x7f04a00c97a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.140+0000 7f04a8479640 1 --2- 172.21.15.131:0/2688223741 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04a00abb40 0x7f04a006b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.140+0000 7f04a8479640 1 -- 172.21.15.131:0/2688223741 >> 172.21.15.131:0/2688223741 conn(0x7f04a0069840 msgr2=0x7f04a00cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:18.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.140+0000 7f04a8479640 1 -- 172.21.15.131:0/2688223741 shutdown_connections 2023-09-12T14:56:18.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.140+0000 7f04a8479640 1 -- 172.21.15.131:0/2688223741 wait complete. 2023-09-12T14:56:18.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.140+0000 7f04a8479640 1 Processor -- start 2023-09-12T14:56:18.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.140+0000 7f04a8479640 1 -- start start 2023-09-12T14:56:18.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.140+0000 7f04a8479640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04a00abb40 0x7f04a014c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:18.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.140+0000 7f04a8479640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04a00c9370 0x7f04a01519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:18.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.140+0000 7f04a8479640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04a0156930 0x7f04a0151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:18.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.140+0000 7f04a8479640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a00accc0 con 0x7f04a00abb40 2023-09-12T14:56:18.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.140+0000 7f04a8479640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a015c320 con 0x7f04a00c9370 2023-09-12T14:56:18.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.140+0000 7f04a8479640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a014cca0 con 0x7f04a0156930 2023-09-12T14:56:18.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.140+0000 7f04a6215640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04a00abb40 0x7f04a014c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:18.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.140+0000 7f04a6215640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04a00abb40 0x7f04a014c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49250/0 (socket says 172.21.15.131:49250) 2023-09-12T14:56:18.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.144+0000 7f04a6215640 1 -- 172.21.15.131:0/1468506424 learned_addr learned my addr 172.21.15.131:0/1468506424 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:18.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.144+0000 7f04a5a14640 1 --2- 172.21.15.131:0/1468506424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04a00c9370 0x7f04a01519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:18.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.144+0000 7f04a6a16640 1 --2- 172.21.15.131:0/1468506424 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04a0156930 0x7f04a0151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:18.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.144+0000 7f04a6215640 1 -- 172.21.15.131:0/1468506424 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04a0156930 msgr2=0x7f04a0151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:18.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.144+0000 7f04a6215640 1 --2- 172.21.15.131:0/1468506424 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04a0156930 0x7f04a0151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.144+0000 7f04a6215640 1 -- 172.21.15.131:0/1468506424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04a00c9370 msgr2=0x7f04a01519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:18.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.144+0000 7f04a6215640 1 --2- 172.21.15.131:0/1468506424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04a00c9370 0x7f04a01519c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.144+0000 7f04a6215640 1 -- 172.21.15.131:0/1468506424 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04940098e0 con 0x7f04a00abb40 2023-09-12T14:56:18.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.144+0000 7f04a5a14640 1 --2- 172.21.15.131:0/1468506424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04a00c9370 0x7f04a01519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:56:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.144+0000 7f04a6215640 1 --2- 172.21.15.131:0/1468506424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04a00abb40 0x7f04a014c760 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f049c001ec0 tx=0x7f049c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.144+0000 7f0492ffd640 1 -- 172.21.15.131:0/1468506424 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c0054e0 con 0x7f04a00abb40 2023-09-12T14:56:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.144+0000 7f0492ffd640 1 -- 172.21.15.131:0/1468506424 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f049c003600 con 0x7f04a00abb40 2023-09-12T14:56:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.144+0000 7f04a8479640 1 -- 172.21.15.131:0/1468506424 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04980094a0 con 0x7f04a00abb40 2023-09-12T14:56:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.144+0000 7f0492ffd640 1 -- 172.21.15.131:0/1468506424 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c003a10 con 0x7f04a00abb40 2023-09-12T14:56:18.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.144+0000 7f04a8479640 1 -- 172.21.15.131:0/1468506424 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a00ac670 con 0x7f04a00abb40 2023-09-12T14:56:18.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.144+0000 7f04a8479640 1 -- 172.21.15.131:0/1468506424 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a0073580 con 0x7f04a00abb40 2023-09-12T14:56:18.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.144+0000 7f0492ffd640 1 -- 172.21.15.131:0/1468506424 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f049c003b70 con 0x7f04a00abb40 2023-09-12T14:56:18.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.144+0000 7f0492ffd640 1 --2- 172.21.15.131:0/1468506424 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0484065b50 0x7f0484068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:18.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.144+0000 7f0492ffd640 1 -- 172.21.15.131:0/1468506424 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f049c082a90 con 0x7f04a00abb40 2023-09-12T14:56:18.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.144+0000 7f04a5a14640 1 --2- 172.21.15.131:0/1468506424 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0484065b50 0x7f0484068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:18.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.144+0000 7f04a5a14640 1 --2- 172.21.15.131:0/1468506424 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0484065b50 0x7f0484068010 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f04a01f7240 tx=0x7f04940095c0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:18.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.148+0000 7f0492ffd640 1 -- 172.21.15.131:0/1468506424 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f049c050350 con 0x7f04a00abb40 2023-09-12T14:56:18.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.328+0000 7f04a8479640 1 -- 172.21.15.131:0/1468506424 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04a00cd4d0 con 0x7f04a00abb40 2023-09-12T14:56:18.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.328+0000 7f0492ffd640 1 -- 172.21.15.131:0/1468506424 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f049c001130 con 0x7f04a00abb40 2023-09-12T14:56:18.331 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:56:18.331 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4676,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/3890238785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3890238785},{"type":"v1","addr":"172.21.15.187:6834","nonce":3890238785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4978,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1436257801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":1436257801},{"type":"v1","addr":"172.21.15.187:6835","nonce":1436257801}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4990,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/806466310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":806466310},{"type":"v1","addr":"172.21.15.131:6837","nonce":806466310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-12T14:55:56.870775+0000","modified":"2023-09-12T14:55:58.869201+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/871788191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":871788191},{"type":"v1","addr":"172.21.15.131:6835","nonce":871788191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-09-12T14:56:18.331 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 13 2023-09-12T14:56:18.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.328+0000 7f04a8479640 1 -- 172.21.15.131:0/1468506424 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0484065b50 msgr2=0x7f0484068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:18.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.328+0000 7f04a8479640 1 --2- 172.21.15.131:0/1468506424 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0484065b50 0x7f0484068010 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f04a01f7240 tx=0x7f04940095c0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.328+0000 7f04a8479640 1 -- 172.21.15.131:0/1468506424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04a00abb40 msgr2=0x7f04a014c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:18.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.328+0000 7f04a8479640 1 --2- 172.21.15.131:0/1468506424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04a00abb40 0x7f04a014c760 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f049c001ec0 tx=0x7f049c00c8e0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.328+0000 7f04a8479640 1 -- 172.21.15.131:0/1468506424 shutdown_connections 2023-09-12T14:56:18.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.328+0000 7f04a8479640 1 --2- 172.21.15.131:0/1468506424 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0484065b50 0x7f0484068010 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.328+0000 7f04a8479640 1 --2- 172.21.15.131:0/1468506424 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04a0156930 0x7f04a0151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.328+0000 7f04a8479640 1 --2- 172.21.15.131:0/1468506424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04a00c9370 0x7f04a01519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.328+0000 7f04a8479640 1 --2- 172.21.15.131:0/1468506424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04a00abb40 0x7f04a014c760 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.328+0000 7f04a8479640 1 -- 172.21.15.131:0/1468506424 >> 172.21.15.131:0/1468506424 conn(0x7f04a0069840 msgr2=0x7f04a006eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:18.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.332+0000 7f04a8479640 1 -- 172.21.15.131:0/1468506424 shutdown_connections 2023-09-12T14:56:18.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.332+0000 7f04a8479640 1 -- 172.21.15.131:0/1468506424 wait complete. 2023-09-12T14:56:18.348 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T14:56:18.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf802f640 1 -- 172.21.15.131:0/2557585885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedf00c9a20 msgr2=0x7fedf006b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:18.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf802f640 1 --2- 172.21.15.131:0/2557585885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedf00c9a20 0x7fedf006b430 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fedf006eaf0 tx=0x7fede001c7d0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf802f640 1 -- 172.21.15.131:0/2557585885 shutdown_connections 2023-09-12T14:56:18.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf802f640 1 --2- 172.21.15.131:0/2557585885 >> v2:172.21.15.187:3300/0 conn(0x7fedf00abae0 0x7fedf015ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf802f640 1 --2- 172.21.15.131:0/2557585885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedf00c9a20 0x7fedf006b430 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf802f640 1 -- 172.21.15.131:0/2557585885 >> 172.21.15.131:0/2557585885 conn(0x7fedf0069840 msgr2=0x7fedf00cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:18.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf802f640 1 -- 172.21.15.131:0/2557585885 shutdown_connections 2023-09-12T14:56:18.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf802f640 1 -- 172.21.15.131:0/2557585885 wait complete. 2023-09-12T14:56:18.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf802f640 1 Processor -- start 2023-09-12T14:56:18.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf802f640 1 -- start start 2023-09-12T14:56:18.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf802f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedf00abae0 0x7fedf0142bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:18.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf802f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedf00c9a20 0x7fedf0140ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:18.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf802f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fedf0158d70 0x7fedf0141670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:18.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf802f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf0142a40 con 0x7fedf00abae0 2023-09-12T14:56:18.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf802f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf0152990 con 0x7fedf00c9a20 2023-09-12T14:56:18.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf802f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf0141ec0 con 0x7fedf0158d70 2023-09-12T14:56:18.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf5dcb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedf00abae0 0x7fedf0142bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:18.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf5dcb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedf00abae0 0x7fedf0142bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44822/0 (socket says 172.21.15.131:44822) 2023-09-12T14:56:18.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf5dcb640 1 -- 172.21.15.131:0/880636311 learned_addr learned my addr 172.21.15.131:0/880636311 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:18.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf55ca640 1 --2- 172.21.15.131:0/880636311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedf00c9a20 0x7fedf0140ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:18.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf65cc640 1 --2- 172.21.15.131:0/880636311 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fedf0158d70 0x7fedf0141670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:18.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf5dcb640 1 -- 172.21.15.131:0/880636311 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fedf0158d70 msgr2=0x7fedf0141670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:18.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf5dcb640 1 --2- 172.21.15.131:0/880636311 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fedf0158d70 0x7fedf0141670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf5dcb640 1 -- 172.21.15.131:0/880636311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedf00c9a20 msgr2=0x7fedf0140ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:18.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf5dcb640 1 --2- 172.21.15.131:0/880636311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedf00c9a20 0x7fedf0140ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf5dcb640 1 -- 172.21.15.131:0/880636311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede0009560 con 0x7fedf00abae0 2023-09-12T14:56:18.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf5dcb640 1 --2- 172.21.15.131:0/880636311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedf00abae0 0x7fedf0142bb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fedec00b3e0 tx=0x7fedec00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:18.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fede67fc640 1 -- 172.21.15.131:0/880636311 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec011830 con 0x7fedf00abae0 2023-09-12T14:56:18.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fede67fc640 1 -- 172.21.15.131:0/880636311 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fedec011aa0 con 0x7fedf00abae0 2023-09-12T14:56:18.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf802f640 1 -- 172.21.15.131:0/880636311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf01421a0 con 0x7fedf00abae0 2023-09-12T14:56:18.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fede67fc640 1 -- 172.21.15.131:0/880636311 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec010380 con 0x7fedf00abae0 2023-09-12T14:56:18.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.456+0000 7fedf802f640 1 -- 172.21.15.131:0/880636311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf0142760 con 0x7fedf00abae0 2023-09-12T14:56:18.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.460+0000 7fedf802f640 1 -- 172.21.15.131:0/880636311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedf0073580 con 0x7fedf00abae0 2023-09-12T14:56:18.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.460+0000 7fede67fc640 1 -- 172.21.15.131:0/880636311 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fedec0104e0 con 0x7fedf00abae0 2023-09-12T14:56:18.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.460+0000 7fede67fc640 1 --2- 172.21.15.131:0/880636311 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fedd4065b20 0x7fedd4067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:18.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.460+0000 7fede67fc640 1 -- 172.21.15.131:0/880636311 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fedec083730 con 0x7fedf00abae0 2023-09-12T14:56:18.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.460+0000 7fedf55ca640 1 --2- 172.21.15.131:0/880636311 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fedd4065b20 0x7fedd4067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:18.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.460+0000 7fedf55ca640 1 --2- 172.21.15.131:0/880636311 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fedd4065b20 0x7fedd4067fe0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fede00ae000 tx=0x7fede001b010 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:18.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.464+0000 7fede67fc640 1 -- 172.21.15.131:0/880636311 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fedec051080 con 0x7fedf00abae0 2023-09-12T14:56:18.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.636+0000 7fedf802f640 1 -- 172.21.15.131:0/880636311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fedf00cd4d0 con 0x7fedf00abae0 2023-09-12T14:56:18.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.640+0000 7fede67fc640 1 -- 172.21.15.131:0/880636311 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fedec0546a0 con 0x7fedf00abae0 2023-09-12T14:56:18.641 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:56:18.642 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4676,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/3890238785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3890238785},{"type":"v1","addr":"172.21.15.187:6834","nonce":3890238785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4978,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1436257801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":1436257801},{"type":"v1","addr":"172.21.15.187:6835","nonce":1436257801}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4990,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/806466310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":806466310},{"type":"v1","addr":"172.21.15.131:6837","nonce":806466310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-12T14:55:56.870775+0000","modified":"2023-09-12T14:55:58.869201+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/871788191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":871788191},{"type":"v1","addr":"172.21.15.131:6835","nonce":871788191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-09-12T14:56:18.642 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 13 2023-09-12T14:56:18.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.640+0000 7fedf802f640 1 -- 172.21.15.131:0/880636311 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fedd4065b20 msgr2=0x7fedd4067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:18.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.640+0000 7fedf802f640 1 --2- 172.21.15.131:0/880636311 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fedd4065b20 0x7fedd4067fe0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fede00ae000 tx=0x7fede001b010 comp rx=0 tx=0).stop 2023-09-12T14:56:18.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.640+0000 7fedf802f640 1 -- 172.21.15.131:0/880636311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedf00abae0 msgr2=0x7fedf0142bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:18.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.640+0000 7fedf802f640 1 --2- 172.21.15.131:0/880636311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedf00abae0 0x7fedf0142bb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fedec00b3e0 tx=0x7fedec00b8a0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.640+0000 7fedf802f640 1 -- 172.21.15.131:0/880636311 shutdown_connections 2023-09-12T14:56:18.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.640+0000 7fedf802f640 1 --2- 172.21.15.131:0/880636311 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fedd4065b20 0x7fedd4067fe0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.640+0000 7fedf802f640 1 --2- 172.21.15.131:0/880636311 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fedf0158d70 0x7fedf0141670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.640+0000 7fedf802f640 1 --2- 172.21.15.131:0/880636311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedf00c9a20 0x7fedf0140ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.640+0000 7fedf802f640 1 --2- 172.21.15.131:0/880636311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedf00abae0 0x7fedf0142bb0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.640+0000 7fedf802f640 1 -- 172.21.15.131:0/880636311 >> 172.21.15.131:0/880636311 conn(0x7fedf0069840 msgr2=0x7fedf014c190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:18.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.640+0000 7fedf802f640 1 -- 172.21.15.131:0/880636311 shutdown_connections 2023-09-12T14:56:18.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.640+0000 7fedf802f640 1 -- 172.21.15.131:0/880636311 wait complete. 2023-09-12T14:56:18.660 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:56:18.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.772+0000 7fe6d33a7640 1 -- 172.21.15.131:0/1447956099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6cc0cf2d0 msgr2=0x7fe6cc15e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:18.776 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.772+0000 7fe6d33a7640 1 --2- 172.21.15.131:0/1447956099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6cc0cf2d0 0x7fe6cc15e2e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009900 tx=0x7fe6c801c880 comp rx=0 tx=0).stop 2023-09-12T14:56:18.776 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.772+0000 7fe6d33a7640 1 -- 172.21.15.131:0/1447956099 shutdown_connections 2023-09-12T14:56:18.776 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.772+0000 7fe6d33a7640 1 --2- 172.21.15.131:0/1447956099 >> v2:172.21.15.187:3300/0 conn(0x7fe6cc0cf6b0 0x7fe6cc0aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.776 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.772+0000 7fe6d33a7640 1 --2- 172.21.15.131:0/1447956099 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6cc0d1680 0x7fe6cc15e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.772+0000 7fe6d33a7640 1 --2- 172.21.15.131:0/1447956099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6cc0cf2d0 0x7fe6cc15e2e0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.772+0000 7fe6d33a7640 1 -- 172.21.15.131:0/1447956099 >> 172.21.15.131:0/1447956099 conn(0x7fe6cc09a200 msgr2=0x7fe6cc0d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:18.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.772+0000 7fe6d33a7640 1 -- 172.21.15.131:0/1447956099 shutdown_connections 2023-09-12T14:56:18.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.772+0000 7fe6d33a7640 1 -- 172.21.15.131:0/1447956099 wait complete. 2023-09-12T14:56:18.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.772+0000 7fe6d33a7640 1 Processor -- start 2023-09-12T14:56:18.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.772+0000 7fe6d33a7640 1 -- start start 2023-09-12T14:56:18.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6d33a7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6cc0cf2d0 0x7fe6cc15d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:18.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6d33a7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6cc0cf6b0 0x7fe6cc0cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:18.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6d33a7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6cc0d1680 0x7fe6cc1543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:18.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6d33a7640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc0b4540 con 0x7fe6cc0cf6b0 2023-09-12T14:56:18.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6d33a7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc0b5b90 con 0x7fe6cc0cf2d0 2023-09-12T14:56:18.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6d33a7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc0b4780 con 0x7fe6cc0d1680 2023-09-12T14:56:18.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6d0942640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6cc0cf6b0 0x7fe6cc0cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:18.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6d0942640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6cc0cf6b0 0x7fe6cc0cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44840/0 (socket says 172.21.15.131:44840) 2023-09-12T14:56:18.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6d0942640 1 -- 172.21.15.131:0/949923213 learned_addr learned my addr 172.21.15.131:0/949923213 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:18.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6d1143640 1 --2- 172.21.15.131:0/949923213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6cc0cf2d0 0x7fe6cc15d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:18.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6d1944640 1 --2- 172.21.15.131:0/949923213 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6cc0d1680 0x7fe6cc1543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:18.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6d0942640 1 -- 172.21.15.131:0/949923213 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6cc0d1680 msgr2=0x7fe6cc1543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:18.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6d0942640 1 --2- 172.21.15.131:0/949923213 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6cc0d1680 0x7fe6cc1543a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6d0942640 1 -- 172.21.15.131:0/949923213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6cc0cf2d0 msgr2=0x7fe6cc15d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:18.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6d0942640 1 --2- 172.21.15.131:0/949923213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6cc0cf2d0 0x7fe6cc15d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6d0942640 1 -- 172.21.15.131:0/949923213 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c8009560 con 0x7fe6cc0cf6b0 2023-09-12T14:56:18.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6d1944640 1 --2- 172.21.15.131:0/949923213 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6cc0d1680 0x7fe6cc1543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:18.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6d0942640 1 --2- 172.21.15.131:0/949923213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6cc0cf6b0 0x7fe6cc0cc300 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc005d10 tx=0x7fe6bc00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:18.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6c1ffb640 1 -- 172.21.15.131:0/949923213 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc009a30 con 0x7fe6cc0cf6b0 2023-09-12T14:56:18.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6c1ffb640 1 -- 172.21.15.131:0/949923213 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe6bc009c80 con 0x7fe6cc0cf6b0 2023-09-12T14:56:18.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6d33a7640 1 -- 172.21.15.131:0/949923213 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6cc0b4a60 con 0x7fe6cc0cf6b0 2023-09-12T14:56:18.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6c1ffb640 1 -- 172.21.15.131:0/949923213 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc005280 con 0x7fe6cc0cf6b0 2023-09-12T14:56:18.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6d33a7640 1 -- 172.21.15.131:0/949923213 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6cc1548e0 con 0x7fe6cc0cf6b0 2023-09-12T14:56:18.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6d33a7640 1 -- 172.21.15.131:0/949923213 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6cc0b4ca0 con 0x7fe6cc0cf6b0 2023-09-12T14:56:18.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6c1ffb640 1 -- 172.21.15.131:0/949923213 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fe6bc0053e0 con 0x7fe6cc0cf6b0 2023-09-12T14:56:18.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6c1ffb640 1 --2- 172.21.15.131:0/949923213 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe6b0065b50 0x7fe6b0068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:18.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6c1ffb640 1 -- 172.21.15.131:0/949923213 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6bc0833f0 con 0x7fe6cc0cf6b0 2023-09-12T14:56:18.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6d1143640 1 --2- 172.21.15.131:0/949923213 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe6b0065b50 0x7fe6b0068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:18.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.776+0000 7fe6d1143640 1 --2- 172.21.15.131:0/949923213 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe6b0065b50 0x7fe6b0068010 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80af000 tx=0x7fe6c801b0c0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:18.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.780+0000 7fe6c1ffb640 1 -- 172.21.15.131:0/949923213 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fe6cc0b4ca0 con 0x7fe6cc0cf6b0 2023-09-12T14:56:18.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.936+0000 7fe6d33a7640 1 -- 172.21.15.131:0/949923213 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6cc0b4ca0 con 0x7fe6cc0cf6b0 2023-09-12T14:56:18.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.936+0000 7fe6c1ffb640 1 -- 172.21.15.131:0/949923213 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16605 (secure 0 0 0) 0x7fe6cc0b4ca0 con 0x7fe6cc0cf6b0 2023-09-12T14:56:18.940 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:56:18.941 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":33,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:55:57.898819+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-09-12T14:55:55.034593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-09-12T14:55:55.855927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:56:18.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.940+0000 7fe6d33a7640 1 -- 172.21.15.131:0/949923213 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe6b0065b50 msgr2=0x7fe6b0068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:18.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.940+0000 7fe6d33a7640 1 --2- 172.21.15.131:0/949923213 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe6b0065b50 0x7fe6b0068010 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80af000 tx=0x7fe6c801b0c0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.940+0000 7fe6d33a7640 1 -- 172.21.15.131:0/949923213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6cc0cf6b0 msgr2=0x7fe6cc0cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:18.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.940+0000 7fe6d33a7640 1 --2- 172.21.15.131:0/949923213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6cc0cf6b0 0x7fe6cc0cc300 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc005d10 tx=0x7fe6bc00dbb0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.940+0000 7fe6d33a7640 1 -- 172.21.15.131:0/949923213 shutdown_connections 2023-09-12T14:56:18.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.940+0000 7fe6d33a7640 1 --2- 172.21.15.131:0/949923213 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe6b0065b50 0x7fe6b0068010 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.940+0000 7fe6d33a7640 1 --2- 172.21.15.131:0/949923213 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6cc0d1680 0x7fe6cc1543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.940+0000 7fe6d33a7640 1 --2- 172.21.15.131:0/949923213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6cc0cf6b0 0x7fe6cc0cc300 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.940+0000 7fe6d33a7640 1 --2- 172.21.15.131:0/949923213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6cc0cf2d0 0x7fe6cc15d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:18.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.940+0000 7fe6d33a7640 1 -- 172.21.15.131:0/949923213 >> 172.21.15.131:0/949923213 conn(0x7fe6cc09a200 msgr2=0x7fe6cc15b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:18.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.940+0000 7fe6d33a7640 1 -- 172.21.15.131:0/949923213 shutdown_connections 2023-09-12T14:56:18.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:18.940+0000 7fe6d33a7640 1 -- 172.21.15.131:0/949923213 wait complete. 2023-09-12T14:56:18.959 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-12T14:56:19.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff94ae3b640 1 -- 172.21.15.131:0/4253810405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9440abb40 msgr2=0x7ff94406b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:19.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff94ae3b640 1 --2- 172.21.15.131:0/4253810405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9440abb40 0x7ff94406b4a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff94406d170 tx=0x7ff93801c870 comp rx=0 tx=0).stop 2023-09-12T14:56:19.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff94ae3b640 1 -- 172.21.15.131:0/4253810405 shutdown_connections 2023-09-12T14:56:19.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff94ae3b640 1 --2- 172.21.15.131:0/4253810405 >> v2:172.21.15.187:3300/0 conn(0x7ff9440c9370 0x7ff9440c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:19.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff94ae3b640 1 --2- 172.21.15.131:0/4253810405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9440abb40 0x7ff94406b4a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:19.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff94ae3b640 1 --2- 172.21.15.131:0/4253810405 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff944156930 0x7ff944159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:19.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff94ae3b640 1 -- 172.21.15.131:0/4253810405 >> 172.21.15.131:0/4253810405 conn(0x7ff944069840 msgr2=0x7ff9440cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:19.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff94ae3b640 1 -- 172.21.15.131:0/4253810405 shutdown_connections 2023-09-12T14:56:19.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff94ae3b640 1 -- 172.21.15.131:0/4253810405 wait complete. 2023-09-12T14:56:19.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff94ae3b640 1 Processor -- start 2023-09-12T14:56:19.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff94ae3b640 1 -- start start 2023-09-12T14:56:19.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff94ae3b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9440abb40 0x7ff94414bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:19.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff94ae3b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9440c9370 0x7ff9440ae710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:19.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff94ae3b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff944156930 0x7ff9440aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:19.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff94ae3b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9440cc700 con 0x7ff9440abb40 2023-09-12T14:56:19.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff94ae3b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9440cc9e0 con 0x7ff944156930 2023-09-12T14:56:19.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff94ae3b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9440cccc0 con 0x7ff9440c9370 2023-09-12T14:56:19.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff948bd7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9440abb40 0x7ff94414bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:19.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff948bd7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9440abb40 0x7ff94414bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44858/0 (socket says 172.21.15.131:44858) 2023-09-12T14:56:19.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff948bd7640 1 -- 172.21.15.131:0/748803914 learned_addr learned my addr 172.21.15.131:0/748803914 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:19.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff9493d8640 1 --2- 172.21.15.131:0/748803914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff944156930 0x7ff9440aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:19.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff943fff640 1 --2- 172.21.15.131:0/748803914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9440c9370 0x7ff9440ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:19.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff948bd7640 1 -- 172.21.15.131:0/748803914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9440c9370 msgr2=0x7ff9440ae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:19.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff948bd7640 1 --2- 172.21.15.131:0/748803914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9440c9370 0x7ff9440ae710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:19.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff948bd7640 1 -- 172.21.15.131:0/748803914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff944156930 msgr2=0x7ff9440aec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:19.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff948bd7640 1 --2- 172.21.15.131:0/748803914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff944156930 0x7ff9440aec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:19.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff948bd7640 1 -- 172.21.15.131:0/748803914 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff938009560 con 0x7ff9440abb40 2023-09-12T14:56:19.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff943fff640 1 --2- 172.21.15.131:0/748803914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9440c9370 0x7ff9440ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:19.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff948bd7640 1 --2- 172.21.15.131:0/748803914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9440abb40 0x7ff94414bb80 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff93400b410 tx=0x7ff93400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:19.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff9417fa640 1 -- 172.21.15.131:0/748803914 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff934011830 con 0x7ff9440abb40 2023-09-12T14:56:19.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff9417fa640 1 -- 172.21.15.131:0/748803914 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff934011a80 con 0x7ff9440abb40 2023-09-12T14:56:19.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff9417fa640 1 -- 172.21.15.131:0/748803914 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff934011e90 con 0x7ff9440abb40 2023-09-12T14:56:19.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff94ae3b640 1 -- 172.21.15.131:0/748803914 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9440adf10 con 0x7ff9440abb40 2023-09-12T14:56:19.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.076+0000 7ff94ae3b640 1 -- 172.21.15.131:0/748803914 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9440adc00 con 0x7ff9440abb40 2023-09-12T14:56:19.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.080+0000 7ff94ae3b640 1 -- 172.21.15.131:0/748803914 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9440ccee0 con 0x7ff9440abb40 2023-09-12T14:56:19.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.080+0000 7ff9417fa640 1 -- 172.21.15.131:0/748803914 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7ff9340193b0 con 0x7ff9440abb40 2023-09-12T14:56:19.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.080+0000 7ff9417fa640 1 --2- 172.21.15.131:0/748803914 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7ff930065b20 0x7ff930067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:19.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.080+0000 7ff9417fa640 1 -- 172.21.15.131:0/748803914 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9340833e0 con 0x7ff9440abb40 2023-09-12T14:56:19.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.080+0000 7ff943fff640 1 --2- 172.21.15.131:0/748803914 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7ff930065b20 0x7ff930067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:19.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.080+0000 7ff943fff640 1 --2- 172.21.15.131:0/748803914 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7ff930065b20 0x7ff930067fe0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff9380af000 tx=0x7ff93801b0b0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:19.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.084+0000 7ff9417fa640 1 -- 172.21.15.131:0/748803914 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7ff9440ccee0 con 0x7ff9440abb40 2023-09-12T14:56:19.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.256+0000 7ff94ae3b640 1 -- 172.21.15.131:0/748803914 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff944073580 con 0x7ff9440abb40 2023-09-12T14:56:19.261 INFO:tasks.ceph.mon.a.smithi131.stderr:2023-09-12T14:56:19.256+0000 7fdecf213640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-12T14:56:19.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.260+0000 7ff9417fa640 1 -- 172.21.15.131:0/748803914 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v14) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff934050c90 con 0x7ff9440abb40 2023-09-12T14:56:19.264 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-12T14:56:19.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.264+0000 7ff94ae3b640 1 -- 172.21.15.131:0/748803914 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7ff930065b20 msgr2=0x7ff930067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:19.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.264+0000 7ff94ae3b640 1 --2- 172.21.15.131:0/748803914 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7ff930065b20 0x7ff930067fe0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff9380af000 tx=0x7ff93801b0b0 comp rx=0 tx=0).stop 2023-09-12T14:56:19.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.264+0000 7ff94ae3b640 1 -- 172.21.15.131:0/748803914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9440abb40 msgr2=0x7ff94414bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:19.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.264+0000 7ff94ae3b640 1 --2- 172.21.15.131:0/748803914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9440abb40 0x7ff94414bb80 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff93400b410 tx=0x7ff93400b8d0 comp rx=0 tx=0).stop 2023-09-12T14:56:19.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.264+0000 7ff94ae3b640 1 -- 172.21.15.131:0/748803914 shutdown_connections 2023-09-12T14:56:19.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.264+0000 7ff94ae3b640 1 --2- 172.21.15.131:0/748803914 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7ff930065b20 0x7ff930067fe0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:19.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.264+0000 7ff94ae3b640 1 --2- 172.21.15.131:0/748803914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9440c9370 0x7ff9440ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:19.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.264+0000 7ff94ae3b640 1 --2- 172.21.15.131:0/748803914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff944156930 0x7ff9440aec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:19.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.264+0000 7ff94ae3b640 1 --2- 172.21.15.131:0/748803914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9440abb40 0x7ff94414bb80 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:19.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.264+0000 7ff94ae3b640 1 -- 172.21.15.131:0/748803914 >> 172.21.15.131:0/748803914 conn(0x7ff944069840 msgr2=0x7ff944159b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:19.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.264+0000 7ff94ae3b640 1 -- 172.21.15.131:0/748803914 shutdown_connections 2023-09-12T14:56:19.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.264+0000 7ff94ae3b640 1 -- 172.21.15.131:0/748803914 wait complete. 2023-09-12T14:56:19.270 INFO:tasks.ceph.mds.a.smithi131.stderr: -1497> 2023-09-12T14:56:02.240+0000 7f656f5ec640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-12T14:56:19.285 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-09-12T14:56:19.298 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2023-09-12T14:56:19.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66db6da640 1 -- 172.21.15.131:0/3664489385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66d40ab730 msgr2=0x7f66d40abb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:19.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66db6da640 1 --2- 172.21.15.131:0/3664489385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66d40ab730 0x7f66d40abb10 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f66d406cd90 tx=0x7f66d001c870 comp rx=0 tx=0).stop 2023-09-12T14:56:19.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66db6da640 1 -- 172.21.15.131:0/3664489385 shutdown_connections 2023-09-12T14:56:19.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66db6da640 1 --2- 172.21.15.131:0/3664489385 >> v2:172.21.15.187:3300/0 conn(0x7f66d406f390 0x7f66d406f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:19.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66db6da640 1 --2- 172.21.15.131:0/3664489385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66d4158180 0x7f66d40cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:19.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66db6da640 1 --2- 172.21.15.131:0/3664489385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66d40ab730 0x7f66d40abb10 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:19.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66db6da640 1 -- 172.21.15.131:0/3664489385 >> 172.21.15.131:0/3664489385 conn(0x7f66d4069840 msgr2=0x7f66d40cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:19.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66db6da640 1 -- 172.21.15.131:0/3664489385 shutdown_connections 2023-09-12T14:56:19.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66db6da640 1 -- 172.21.15.131:0/3664489385 wait complete. 2023-09-12T14:56:19.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66db6da640 1 Processor -- start 2023-09-12T14:56:19.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66db6da640 1 -- start start 2023-09-12T14:56:19.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66db6da640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66d406f390 0x7f66d40b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:19.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66db6da640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66d40ab730 0x7f66d4149010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:19.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66db6da640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66d4158180 0x7f66d41f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:19.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66db6da640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d4148e80 con 0x7f66d40ab730 2023-09-12T14:56:19.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66db6da640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d4149fb0 con 0x7f66d4158180 2023-09-12T14:56:19.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66db6da640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d4149710 con 0x7f66d406f390 2023-09-12T14:56:19.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66d8c75640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66d40ab730 0x7f66d4149010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:19.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66d8c75640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66d40ab730 0x7f66d4149010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44908/0 (socket says 172.21.15.131:44908) 2023-09-12T14:56:19.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66d8c75640 1 -- 172.21.15.131:0/357315932 learned_addr learned my addr 172.21.15.131:0/357315932 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:19.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66d8c75640 1 -- 172.21.15.131:0/357315932 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66d406f390 msgr2=0x7f66d40b1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:19.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66d8c75640 1 --2- 172.21.15.131:0/357315932 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66d406f390 0x7f66d40b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:19.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66d9c77640 1 --2- 172.21.15.131:0/357315932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66d4158180 0x7f66d41f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:19.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66d8c75640 1 -- 172.21.15.131:0/357315932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66d4158180 msgr2=0x7f66d41f57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:19.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66d8c75640 1 --2- 172.21.15.131:0/357315932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66d4158180 0x7f66d41f57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:19.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66d8c75640 1 -- 172.21.15.131:0/357315932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d0009560 con 0x7f66d40ab730 2023-09-12T14:56:19.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66d8c75640 1 --2- 172.21.15.131:0/357315932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66d40ab730 0x7f66d4149010 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f66c400c9f0 tx=0x7f66c400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:19.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66c9ffb640 1 -- 172.21.15.131:0/357315932 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c4007bb0 con 0x7f66d40ab730 2023-09-12T14:56:19.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66c9ffb640 1 -- 172.21.15.131:0/357315932 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f66c4007e00 con 0x7f66d40ab730 2023-09-12T14:56:19.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66c9ffb640 1 -- 172.21.15.131:0/357315932 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c400f6b0 con 0x7f66d40ab730 2023-09-12T14:56:19.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66db6da640 1 -- 172.21.15.131:0/357315932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d41499f0 con 0x7f66d40ab730 2023-09-12T14:56:19.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.400+0000 7f66db6da640 1 -- 172.21.15.131:0/357315932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d414a2e0 con 0x7f66d40ab730 2023-09-12T14:56:19.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.404+0000 7f66db6da640 1 -- 172.21.15.131:0/357315932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66d4073580 con 0x7f66d40ab730 2023-09-12T14:56:19.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.404+0000 7f66c9ffb640 1 -- 172.21.15.131:0/357315932 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f66c400f810 con 0x7f66d40ab730 2023-09-12T14:56:19.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.404+0000 7f66c9ffb640 1 --2- 172.21.15.131:0/357315932 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f66b8065b20 0x7f66b8067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:19.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.404+0000 7f66d9476640 1 --2- 172.21.15.131:0/357315932 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f66b8065b20 0x7f66b8067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:19.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.404+0000 7f66c9ffb640 1 -- 172.21.15.131:0/357315932 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f66c4083ad0 con 0x7f66d40ab730 2023-09-12T14:56:19.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.404+0000 7f66d9476640 1 --2- 172.21.15.131:0/357315932 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f66b8065b20 0x7f66b8067fe0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f66d00af000 tx=0x7f66d0019610 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:19.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.408+0000 7f66c9ffb640 1 -- 172.21.15.131:0/357315932 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f66c4051450 con 0x7f66d40ab730 2023-09-12T14:56:19.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:19.588+0000 7f66db6da640 1 -- 172.21.15.131:0/357315932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f66d40ae690 con 0x7f66d40ab730 2023-09-12T14:56:20.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.272+0000 7f66c9ffb640 1 -- 172.21.15.131:0/357315932 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f66c4054a70 con 0x7f66d40ab730 2023-09-12T14:56:20.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.276+0000 7f66db6da640 1 -- 172.21.15.131:0/357315932 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f66b8065b20 msgr2=0x7f66b8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:20.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.276+0000 7f66db6da640 1 --2- 172.21.15.131:0/357315932 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f66b8065b20 0x7f66b8067fe0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f66d00af000 tx=0x7f66d0019610 comp rx=0 tx=0).stop 2023-09-12T14:56:20.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.276+0000 7f66db6da640 1 -- 172.21.15.131:0/357315932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66d40ab730 msgr2=0x7f66d4149010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:20.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.276+0000 7f66db6da640 1 --2- 172.21.15.131:0/357315932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66d40ab730 0x7f66d4149010 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f66c400c9f0 tx=0x7f66c400ceb0 comp rx=0 tx=0).stop 2023-09-12T14:56:20.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.276+0000 7f66db6da640 1 -- 172.21.15.131:0/357315932 shutdown_connections 2023-09-12T14:56:20.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.276+0000 7f66db6da640 1 --2- 172.21.15.131:0/357315932 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f66b8065b20 0x7f66b8067fe0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:20.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.276+0000 7f66db6da640 1 --2- 172.21.15.131:0/357315932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66d4158180 0x7f66d41f57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:20.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.276+0000 7f66db6da640 1 --2- 172.21.15.131:0/357315932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66d40ab730 0x7f66d4149010 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:20.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.276+0000 7f66db6da640 1 --2- 172.21.15.131:0/357315932 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66d406f390 0x7f66d40b1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:20.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.276+0000 7f66db6da640 1 -- 172.21.15.131:0/357315932 >> 172.21.15.131:0/357315932 conn(0x7f66d4069840 msgr2=0x7f66d40aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:20.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.276+0000 7f66db6da640 1 -- 172.21.15.131:0/357315932 shutdown_connections 2023-09-12T14:56:20.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.276+0000 7f66db6da640 1 -- 172.21.15.131:0/357315932 wait complete. 2023-09-12T14:56:20.304 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-09-12T14:56:20.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.480+0000 7eff88859640 1 -- 172.21.15.131:0/1162711060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff840cf2d0 msgr2=0x7eff8415e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:20.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.480+0000 7eff88859640 1 --2- 172.21.15.131:0/1162711060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff840cf2d0 0x7eff8415e2e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7eff84089db0 tx=0x7eff7801c7e0 comp rx=0 tx=0).stop 2023-09-12T14:56:20.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.480+0000 7eff88859640 1 -- 172.21.15.131:0/1162711060 shutdown_connections 2023-09-12T14:56:20.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.480+0000 7eff88859640 1 --2- 172.21.15.131:0/1162711060 >> v2:172.21.15.187:3300/0 conn(0x7eff840d1680 0x7eff8415e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:20.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.480+0000 7eff88859640 1 --2- 172.21.15.131:0/1162711060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff840cf2d0 0x7eff8415e2e0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:20.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.480+0000 7eff88859640 1 -- 172.21.15.131:0/1162711060 >> 172.21.15.131:0/1162711060 conn(0x7eff8409a200 msgr2=0x7eff840d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:20.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.480+0000 7eff88859640 1 -- 172.21.15.131:0/1162711060 shutdown_connections 2023-09-12T14:56:20.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.480+0000 7eff88859640 1 -- 172.21.15.131:0/1162711060 wait complete. 2023-09-12T14:56:20.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.484+0000 7eff88859640 1 Processor -- start 2023-09-12T14:56:20.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.484+0000 7eff88859640 1 -- start start 2023-09-12T14:56:20.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.484+0000 7eff88859640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff840cf2d0 0x7eff840cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:20.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.484+0000 7eff88859640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7eff840cf6b0 0x7eff8415b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:20.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.484+0000 7eff88859640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff840d1680 0x7eff8415bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:20.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.484+0000 7eff88859640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff840b3de0 con 0x7eff840d1680 2023-09-12T14:56:20.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.484+0000 7eff88859640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff84156550 con 0x7eff840cf2d0 2023-09-12T14:56:20.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.484+0000 7eff88859640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff840c9760 con 0x7eff840cf6b0 2023-09-12T14:56:20.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.484+0000 7eff82d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff840d1680 0x7eff8415bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:20.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.484+0000 7eff82d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff840d1680 0x7eff8415bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44930/0 (socket says 172.21.15.131:44930) 2023-09-12T14:56:20.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.484+0000 7eff82d9d640 1 -- 172.21.15.131:0/763265705 learned_addr learned my addr 172.21.15.131:0/763265705 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:20.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.484+0000 7eff8259c640 1 --2- 172.21.15.131:0/763265705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff840cf2d0 0x7eff840cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:20.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.484+0000 7eff81d9b640 1 --2- 172.21.15.131:0/763265705 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7eff840cf6b0 0x7eff8415b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:20.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.484+0000 7eff82d9d640 1 -- 172.21.15.131:0/763265705 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7eff840cf6b0 msgr2=0x7eff8415b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:20.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.484+0000 7eff82d9d640 1 --2- 172.21.15.131:0/763265705 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7eff840cf6b0 0x7eff8415b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:20.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.484+0000 7eff82d9d640 1 -- 172.21.15.131:0/763265705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff840cf2d0 msgr2=0x7eff840cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:20.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.484+0000 7eff82d9d640 1 --2- 172.21.15.131:0/763265705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff840cf2d0 0x7eff840cd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:20.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.484+0000 7eff82d9d640 1 -- 172.21.15.131:0/763265705 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff780094a0 con 0x7eff840d1680 2023-09-12T14:56:20.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.484+0000 7eff82d9d640 1 --2- 172.21.15.131:0/763265705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff840d1680 0x7eff8415bc10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00cbc0 tx=0x7eff7c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:20.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.484+0000 7eff72ffd640 1 -- 172.21.15.131:0/763265705 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c00aa50 con 0x7eff840d1680 2023-09-12T14:56:20.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.484+0000 7eff72ffd640 1 -- 172.21.15.131:0/763265705 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7eff7c00aca0 con 0x7eff840d1680 2023-09-12T14:56:20.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.484+0000 7eff88859640 1 -- 172.21.15.131:0/763265705 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff840c9a60 con 0x7eff840d1680 2023-09-12T14:56:20.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.484+0000 7eff72ffd640 1 -- 172.21.15.131:0/763265705 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c012680 con 0x7eff840d1680 2023-09-12T14:56:20.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.484+0000 7eff88859640 1 -- 172.21.15.131:0/763265705 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff840cc820 con 0x7eff840d1680 2023-09-12T14:56:20.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.484+0000 7eff88859640 1 -- 172.21.15.131:0/763265705 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff840caf50 con 0x7eff840d1680 2023-09-12T14:56:20.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.488+0000 7eff72ffd640 1 -- 172.21.15.131:0/763265705 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7eff7c01a3b0 con 0x7eff840d1680 2023-09-12T14:56:20.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.488+0000 7eff72ffd640 1 --2- 172.21.15.131:0/763265705 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7eff64065b50 0x7eff64068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:20.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.488+0000 7eff72ffd640 1 -- 172.21.15.131:0/763265705 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5795+0+0 (secure 0 0 0) 0x7eff7c083780 con 0x7eff840d1680 2023-09-12T14:56:20.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.488+0000 7eff8259c640 1 --2- 172.21.15.131:0/763265705 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7eff64065b50 0x7eff64068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:20.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.488+0000 7eff8259c640 1 --2- 172.21.15.131:0/763265705 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7eff64065b50 0x7eff64068010 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7eff780af000 tx=0x7eff7801b020 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:20.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.492+0000 7eff72ffd640 1 -- 172.21.15.131:0/763265705 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7eff7c051070 con 0x7eff840d1680 2023-09-12T14:56:20.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:20.648+0000 7eff88859640 1 -- 172.21.15.131:0/763265705 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7eff840b3b60 con 0x7eff840d1680 2023-09-12T14:56:21.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.276+0000 7eff72ffd640 1 -- 172.21.15.131:0/763265705 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) v1 ==== 181+0+0 (secure 0 0 0) 0x7eff7c054690 con 0x7eff840d1680 2023-09-12T14:56:21.280 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2023-09-12T14:56:21.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.280+0000 7eff88859640 1 -- 172.21.15.131:0/763265705 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7eff64065b50 msgr2=0x7eff64068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:21.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.280+0000 7eff88859640 1 --2- 172.21.15.131:0/763265705 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7eff64065b50 0x7eff64068010 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7eff780af000 tx=0x7eff7801b020 comp rx=0 tx=0).stop 2023-09-12T14:56:21.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.280+0000 7eff88859640 1 -- 172.21.15.131:0/763265705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff840d1680 msgr2=0x7eff8415bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:21.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.280+0000 7eff88859640 1 --2- 172.21.15.131:0/763265705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff840d1680 0x7eff8415bc10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00cbc0 tx=0x7eff7c00ccb0 comp rx=0 tx=0).stop 2023-09-12T14:56:21.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.280+0000 7eff88859640 1 -- 172.21.15.131:0/763265705 shutdown_connections 2023-09-12T14:56:21.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.280+0000 7eff88859640 1 --2- 172.21.15.131:0/763265705 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7eff64065b50 0x7eff64068010 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:21.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.280+0000 7eff88859640 1 --2- 172.21.15.131:0/763265705 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7eff840cf6b0 0x7eff8415b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:21.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.280+0000 7eff88859640 1 --2- 172.21.15.131:0/763265705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff840d1680 0x7eff8415bc10 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:21.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.280+0000 7eff88859640 1 --2- 172.21.15.131:0/763265705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff840cf2d0 0x7eff840cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:21.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.280+0000 7eff88859640 1 -- 172.21.15.131:0/763265705 >> 172.21.15.131:0/763265705 conn(0x7eff8409a200 msgr2=0x7eff8415e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:21.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.284+0000 7eff88859640 1 -- 172.21.15.131:0/763265705 shutdown_connections 2023-09-12T14:56:21.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.284+0000 7eff88859640 1 -- 172.21.15.131:0/763265705 wait complete. 2023-09-12T14:56:21.308 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-09-12T14:56:21.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fd15b6640 1 -- 172.21.15.131:0/1967935971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fcc0b2600 msgr2=0x7f2fcc0c9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:21.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fd15b6640 1 --2- 172.21.15.131:0/1967935971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fcc0b2600 0x7f2fcc0c9390 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000aa10 tx=0x7f2fc001dba0 comp rx=0 tx=0).stop 2023-09-12T14:56:21.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fd15b6640 1 -- 172.21.15.131:0/1967935971 shutdown_connections 2023-09-12T14:56:21.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fd15b6640 1 --2- 172.21.15.131:0/1967935971 >> v2:172.21.15.187:3300/0 conn(0x7f2fcc0c98d0 0x7f2fcc0af9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:21.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fd15b6640 1 --2- 172.21.15.131:0/1967935971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fcc0b2600 0x7f2fcc0c9390 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:21.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fd15b6640 1 -- 172.21.15.131:0/1967935971 >> 172.21.15.131:0/1967935971 conn(0x7f2fcc069840 msgr2=0x7f2fcc0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:21.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fd15b6640 1 -- 172.21.15.131:0/1967935971 shutdown_connections 2023-09-12T14:56:21.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fd15b6640 1 -- 172.21.15.131:0/1967935971 wait complete. 2023-09-12T14:56:21.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fd15b6640 1 Processor -- start 2023-09-12T14:56:21.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fd15b6640 1 -- start start 2023-09-12T14:56:21.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fd15b6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fcc0b2600 0x7f2fcc147460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:21.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fd15b6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fcc0c98d0 0x7f2fcc0c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:21.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fd15b6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fcc1574d0 0x7f2fcc0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:21.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fd15b6640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fcc151220 con 0x7f2fcc0b2600 2023-09-12T14:56:21.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fd15b6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fcc146440 con 0x7f2fcc0c98d0 2023-09-12T14:56:21.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fd15b6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fcc147130 con 0x7f2fcc1574d0 2023-09-12T14:56:21.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fcaffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fcc0b2600 0x7f2fcc147460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:21.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fcaffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fcc0b2600 0x7f2fcc147460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44934/0 (socket says 172.21.15.131:44934) 2023-09-12T14:56:21.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fcaffd640 1 -- 172.21.15.131:0/2890582557 learned_addr learned my addr 172.21.15.131:0/2890582557 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:21.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fcb7fe640 1 --2- 172.21.15.131:0/2890582557 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fcc1574d0 0x7f2fcc0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:21.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fca7fc640 1 --2- 172.21.15.131:0/2890582557 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fcc0c98d0 0x7f2fcc0c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:21.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fcaffd640 1 -- 172.21.15.131:0/2890582557 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fcc1574d0 msgr2=0x7f2fcc0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:21.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fcaffd640 1 --2- 172.21.15.131:0/2890582557 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fcc1574d0 0x7f2fcc0ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:21.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fcaffd640 1 -- 172.21.15.131:0/2890582557 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fcc0c98d0 msgr2=0x7f2fcc0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:21.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fcaffd640 1 --2- 172.21.15.131:0/2890582557 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fcc0c98d0 0x7f2fcc0c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:21.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fcaffd640 1 -- 172.21.15.131:0/2890582557 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc000a670 con 0x7f2fcc0b2600 2023-09-12T14:56:21.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fcaffd640 1 --2- 172.21.15.131:0/2890582557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fcc0b2600 0x7f2fcc147460 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00ba50 tx=0x7f2fbc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:21.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fb37fe640 1 -- 172.21.15.131:0/2890582557 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc00cbe0 con 0x7f2fcc0b2600 2023-09-12T14:56:21.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fb37fe640 1 -- 172.21.15.131:0/2890582557 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2fbc00ce70 con 0x7f2fcc0b2600 2023-09-12T14:56:21.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fb37fe640 1 -- 172.21.15.131:0/2890582557 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc014680 con 0x7f2fcc0b2600 2023-09-12T14:56:21.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fd15b6640 1 -- 172.21.15.131:0/2890582557 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fcc146e50 con 0x7f2fcc0b2600 2023-09-12T14:56:21.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.440+0000 7f2fd15b6640 1 -- 172.21.15.131:0/2890582557 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fcc150eb0 con 0x7f2fcc0b2600 2023-09-12T14:56:21.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.444+0000 7f2fd15b6640 1 -- 172.21.15.131:0/2890582557 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fcc073580 con 0x7f2fcc0b2600 2023-09-12T14:56:21.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.444+0000 7f2fb37fe640 1 -- 172.21.15.131:0/2890582557 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f2fbc0147e0 con 0x7f2fcc0b2600 2023-09-12T14:56:21.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.444+0000 7f2fb37fe640 1 --2- 172.21.15.131:0/2890582557 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2fb8065b20 0x7f2fb8067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:21.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.444+0000 7f2fca7fc640 1 --2- 172.21.15.131:0/2890582557 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2fb8065b20 0x7f2fb8067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:21.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.444+0000 7f2fb37fe640 1 -- 172.21.15.131:0/2890582557 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f2fbc0833c0 con 0x7f2fcc0b2600 2023-09-12T14:56:21.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.444+0000 7f2fca7fc640 1 --2- 172.21.15.131:0/2890582557 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2fb8065b20 0x7f2fb8067fe0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2fc00b0000 tx=0x7f2fc001c3e0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:21.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.448+0000 7f2fb37fe640 1 -- 172.21.15.131:0/2890582557 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f2fbc050ea0 con 0x7f2fcc0b2600 2023-09-12T14:56:21.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:21.600+0000 7f2fd15b6640 1 -- 172.21.15.131:0/2890582557 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2fcc14f5c0 con 0x7f2fcc0b2600 2023-09-12T14:56:22.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.280+0000 7f2fb37fe640 1 -- 172.21.15.131:0/2890582557 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2fbc0544c0 con 0x7f2fcc0b2600 2023-09-12T14:56:22.283 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2023-09-12T14:56:22.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.284+0000 7f2fd15b6640 1 -- 172.21.15.131:0/2890582557 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2fb8065b20 msgr2=0x7f2fb8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:22.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.284+0000 7f2fd15b6640 1 --2- 172.21.15.131:0/2890582557 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2fb8065b20 0x7f2fb8067fe0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2fc00b0000 tx=0x7f2fc001c3e0 comp rx=0 tx=0).stop 2023-09-12T14:56:22.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.284+0000 7f2fd15b6640 1 -- 172.21.15.131:0/2890582557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fcc0b2600 msgr2=0x7f2fcc147460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:22.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.284+0000 7f2fd15b6640 1 --2- 172.21.15.131:0/2890582557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fcc0b2600 0x7f2fcc147460 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00ba50 tx=0x7f2fbc00bf10 comp rx=0 tx=0).stop 2023-09-12T14:56:22.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.284+0000 7f2fd15b6640 1 -- 172.21.15.131:0/2890582557 shutdown_connections 2023-09-12T14:56:22.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.284+0000 7f2fd15b6640 1 --2- 172.21.15.131:0/2890582557 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f2fb8065b20 0x7f2fb8067fe0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:22.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.284+0000 7f2fd15b6640 1 --2- 172.21.15.131:0/2890582557 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fcc1574d0 0x7f2fcc0ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:22.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.284+0000 7f2fd15b6640 1 --2- 172.21.15.131:0/2890582557 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fcc0c98d0 0x7f2fcc0c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:22.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.284+0000 7f2fd15b6640 1 --2- 172.21.15.131:0/2890582557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fcc0b2600 0x7f2fcc147460 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:22.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.284+0000 7f2fd15b6640 1 -- 172.21.15.131:0/2890582557 >> 172.21.15.131:0/2890582557 conn(0x7f2fcc069840 msgr2=0x7f2fcc0cc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:22.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.284+0000 7f2fd15b6640 1 -- 172.21.15.131:0/2890582557 shutdown_connections 2023-09-12T14:56:22.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.288+0000 7f2fd15b6640 1 -- 172.21.15.131:0/2890582557 wait complete. 2023-09-12T14:56:22.314 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 30 2023-09-12T14:56:22.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.436+0000 7f1efa497640 1 -- 172.21.15.131:0/981965757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ef40af880 msgr2=0x7f1ef40c9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:22.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.436+0000 7f1efa497640 1 --2- 172.21.15.131:0/981965757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ef40af880 0x7f1ef40c9420 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1edc009900 tx=0x7f1edc01c880 comp rx=0 tx=0).stop 2023-09-12T14:56:22.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.436+0000 7f1efa497640 1 -- 172.21.15.131:0/981965757 shutdown_connections 2023-09-12T14:56:22.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.436+0000 7f1efa497640 1 --2- 172.21.15.131:0/981965757 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ef40c9960 0x7f1ef40cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:22.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.436+0000 7f1efa497640 1 --2- 172.21.15.131:0/981965757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ef40af880 0x7f1ef40c9420 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:22.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.436+0000 7f1efa497640 1 -- 172.21.15.131:0/981965757 >> 172.21.15.131:0/981965757 conn(0x7f1ef40ae580 msgr2=0x7f1ef40ae990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:22.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.436+0000 7f1efa497640 1 -- 172.21.15.131:0/981965757 shutdown_connections 2023-09-12T14:56:22.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.436+0000 7f1efa497640 1 -- 172.21.15.131:0/981965757 wait complete. 2023-09-12T14:56:22.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.436+0000 7f1efa497640 1 Processor -- start 2023-09-12T14:56:22.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.436+0000 7f1efa497640 1 -- start start 2023-09-12T14:56:22.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.440+0000 7f1efa497640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ef40af880 0x7f1ef406b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:22.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.440+0000 7f1efa497640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ef40c9960 0x7f1ef414ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:22.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.440+0000 7f1efa497640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef4158e80 0x7f1ef414f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:22.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.440+0000 7f1efa497640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef4156900 con 0x7f1ef40c9960 2023-09-12T14:56:22.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.440+0000 7f1efa497640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef4156e00 con 0x7f1ef4158e80 2023-09-12T14:56:22.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.440+0000 7f1efa497640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef4150f10 con 0x7f1ef40af880 2023-09-12T14:56:22.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.440+0000 7f1ef37fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ef40c9960 0x7f1ef414ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:22.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.440+0000 7f1ef37fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ef40c9960 0x7f1ef414ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44956/0 (socket says 172.21.15.131:44956) 2023-09-12T14:56:22.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.440+0000 7f1ef37fe640 1 -- 172.21.15.131:0/3583812576 learned_addr learned my addr 172.21.15.131:0/3583812576 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:22.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.440+0000 7f1ef3fff640 1 --2- 172.21.15.131:0/3583812576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ef40af880 0x7f1ef406b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:22.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.440+0000 7f1ef8a34640 1 --2- 172.21.15.131:0/3583812576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef4158e80 0x7f1ef414f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:22.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.440+0000 7f1ef37fe640 1 -- 172.21.15.131:0/3583812576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ef40af880 msgr2=0x7f1ef406b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:22.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.440+0000 7f1ef37fe640 1 --2- 172.21.15.131:0/3583812576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ef40af880 0x7f1ef406b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:22.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.440+0000 7f1ef37fe640 1 -- 172.21.15.131:0/3583812576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef4158e80 msgr2=0x7f1ef414f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:22.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.440+0000 7f1ef37fe640 1 --2- 172.21.15.131:0/3583812576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef4158e80 0x7f1ef414f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:22.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.440+0000 7f1ef37fe640 1 -- 172.21.15.131:0/3583812576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1edc009560 con 0x7f1ef40c9960 2023-09-12T14:56:22.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.440+0000 7f1ef37fe640 1 --2- 172.21.15.131:0/3583812576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ef40c9960 0x7f1ef414ebe0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1edc000c10 tx=0x7f1edc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:22.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.440+0000 7f1ef0ff9640 1 -- 172.21.15.131:0/3583812576 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc005030 con 0x7f1ef40c9960 2023-09-12T14:56:22.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.440+0000 7f1ef0ff9640 1 -- 172.21.15.131:0/3583812576 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1edc005280 con 0x7f1ef40c9960 2023-09-12T14:56:22.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.440+0000 7f1efa497640 1 -- 172.21.15.131:0/3583812576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef41511f0 con 0x7f1ef40c9960 2023-09-12T14:56:22.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.440+0000 7f1ef0ff9640 1 -- 172.21.15.131:0/3583812576 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc0046d0 con 0x7f1ef40c9960 2023-09-12T14:56:22.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.440+0000 7f1efa497640 1 -- 172.21.15.131:0/3583812576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef41517b0 con 0x7f1ef40c9960 2023-09-12T14:56:22.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.440+0000 7f1efa497640 1 -- 172.21.15.131:0/3583812576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef4073580 con 0x7f1ef40c9960 2023-09-12T14:56:22.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.440+0000 7f1ef0ff9640 1 -- 172.21.15.131:0/3583812576 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f1edc0b4040 con 0x7f1ef40c9960 2023-09-12T14:56:22.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.440+0000 7f1ef0ff9640 1 --2- 172.21.15.131:0/3583812576 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1ed4065b20 0x7f1ed4067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:22.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.440+0000 7f1ef0ff9640 1 -- 172.21.15.131:0/3583812576 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1edc11cd20 con 0x7f1ef40c9960 2023-09-12T14:56:22.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.440+0000 7f1ef3fff640 1 --2- 172.21.15.131:0/3583812576 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1ed4065b20 0x7f1ed4067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:22.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.444+0000 7f1ef3fff640 1 --2- 172.21.15.131:0/3583812576 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1ed4065b20 0x7f1ed4067fe0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1eec009f20 tx=0x7f1eec0094d0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:22.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.444+0000 7f1ef0ff9640 1 -- 172.21.15.131:0/3583812576 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f1edc0ea970 con 0x7f1ef40c9960 2023-09-12T14:56:22.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.612+0000 7f1efa497640 1 -- 172.21.15.131:0/3583812576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config reset", "num": 30} v 0) v1 -- 0x7f1ef414ad30 con 0x7f1ef40c9960 2023-09-12T14:56:22.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.616+0000 7f1ef0ff9640 1 -- 172.21.15.131:0/3583812576 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 30}]=0 v32) v1 ==== 71+0+0 (secure 0 0 0) 0x7f1edc0bc080 con 0x7f1ef40c9960 2023-09-12T14:56:22.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.620+0000 7f1efa497640 1 -- 172.21.15.131:0/3583812576 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1ed4065b20 msgr2=0x7f1ed4067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:22.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.620+0000 7f1efa497640 1 --2- 172.21.15.131:0/3583812576 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1ed4065b20 0x7f1ed4067fe0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1eec009f20 tx=0x7f1eec0094d0 comp rx=0 tx=0).stop 2023-09-12T14:56:22.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.620+0000 7f1efa497640 1 -- 172.21.15.131:0/3583812576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ef40c9960 msgr2=0x7f1ef414ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:22.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.620+0000 7f1efa497640 1 --2- 172.21.15.131:0/3583812576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ef40c9960 0x7f1ef414ebe0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1edc000c10 tx=0x7f1edc0040c0 comp rx=0 tx=0).stop 2023-09-12T14:56:22.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.620+0000 7f1efa497640 1 -- 172.21.15.131:0/3583812576 shutdown_connections 2023-09-12T14:56:22.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.620+0000 7f1efa497640 1 --2- 172.21.15.131:0/3583812576 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1ed4065b20 0x7f1ed4067fe0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:22.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.620+0000 7f1efa497640 1 --2- 172.21.15.131:0/3583812576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef4158e80 0x7f1ef414f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:22.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.620+0000 7f1efa497640 1 --2- 172.21.15.131:0/3583812576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ef40c9960 0x7f1ef414ebe0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:22.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.620+0000 7f1efa497640 1 --2- 172.21.15.131:0/3583812576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ef40af880 0x7f1ef406b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:22.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.620+0000 7f1efa497640 1 -- 172.21.15.131:0/3583812576 >> 172.21.15.131:0/3583812576 conn(0x7f1ef40ae580 msgr2=0x7f1ef4152750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:22.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.620+0000 7f1efa497640 1 -- 172.21.15.131:0/3583812576 shutdown_connections 2023-09-12T14:56:22.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.620+0000 7f1efa497640 1 -- 172.21.15.131:0/3583812576 wait complete. 2023-09-12T14:56:22.639 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values' 2023-09-12T14:56:22.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.760+0000 7f856ee3d640 1 -- 172.21.15.131:0/2949870654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85680b1760 msgr2=0x7f856815e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:22.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.760+0000 7f856ee3d640 1 --2- 172.21.15.131:0/2949870654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85680b1760 0x7f856815e820 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8560009cb0 tx=0x7f856001cd40 comp rx=0 tx=0).stop 2023-09-12T14:56:22.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.760+0000 7f856ee3d640 1 -- 172.21.15.131:0/2949870654 shutdown_connections 2023-09-12T14:56:22.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.760+0000 7f856ee3d640 1 --2- 172.21.15.131:0/2949870654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85680b1760 0x7f856815e820 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:22.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.760+0000 7f856ee3d640 1 --2- 172.21.15.131:0/2949870654 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85680cf2d0 0x7f856815e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:22.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.760+0000 7f856ee3d640 1 -- 172.21.15.131:0/2949870654 >> 172.21.15.131:0/2949870654 conn(0x7f856809a200 msgr2=0x7f85680d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:22.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.760+0000 7f856ee3d640 1 -- 172.21.15.131:0/2949870654 shutdown_connections 2023-09-12T14:56:22.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.760+0000 7f856ee3d640 1 -- 172.21.15.131:0/2949870654 wait complete. 2023-09-12T14:56:22.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.760+0000 7f856ee3d640 1 Processor -- start 2023-09-12T14:56:22.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.760+0000 7f856ee3d640 1 -- start start 2023-09-12T14:56:22.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.760+0000 7f856ee3d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85680b1760 0x7f8568151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:22.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.760+0000 7f856ee3d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85680cf2d0 0x7f856815ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:22.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.760+0000 7f856ee3d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85680d1680 0x7f85680cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:22.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.760+0000 7f856ee3d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85680c8da0 con 0x7f85680d1680 2023-09-12T14:56:22.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.760+0000 7f856ee3d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85680c9080 con 0x7f85680b1760 2023-09-12T14:56:22.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.760+0000 7f856ee3d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85680c9360 con 0x7f85680cf2d0 2023-09-12T14:56:22.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.760+0000 7f856d3da640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85680d1680 0x7f85680cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:22.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.760+0000 7f856d3da640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85680d1680 0x7f85680cd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44972/0 (socket says 172.21.15.131:44972) 2023-09-12T14:56:22.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.760+0000 7f856d3da640 1 -- 172.21.15.131:0/796434842 learned_addr learned my addr 172.21.15.131:0/796434842 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:22.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.764+0000 7f856cbd9640 1 --2- 172.21.15.131:0/796434842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85680b1760 0x7f8568151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:22.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.764+0000 7f8567fff640 1 --2- 172.21.15.131:0/796434842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85680cf2d0 0x7f856815ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:22.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.764+0000 7f856d3da640 1 -- 172.21.15.131:0/796434842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85680cf2d0 msgr2=0x7f856815ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:22.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.764+0000 7f856d3da640 1 --2- 172.21.15.131:0/796434842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85680cf2d0 0x7f856815ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:22.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.764+0000 7f856d3da640 1 -- 172.21.15.131:0/796434842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85680b1760 msgr2=0x7f8568151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:22.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.764+0000 7f856d3da640 1 --2- 172.21.15.131:0/796434842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85680b1760 0x7f8568151b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:22.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.764+0000 7f856d3da640 1 -- 172.21.15.131:0/796434842 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85600098e0 con 0x7f85680d1680 2023-09-12T14:56:22.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.764+0000 7f856d3da640 1 --2- 172.21.15.131:0/796434842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85680d1680 0x7f85680cd130 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8560009bd0 tx=0x7f8560003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:22.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.764+0000 7f85657fa640 1 -- 172.21.15.131:0/796434842 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85600ba070 con 0x7f85680d1680 2023-09-12T14:56:22.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.764+0000 7f85657fa640 1 -- 172.21.15.131:0/796434842 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8560004b40 con 0x7f85680d1680 2023-09-12T14:56:22.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.764+0000 7f856ee3d640 1 -- 172.21.15.131:0/796434842 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85680c9640 con 0x7f85680d1680 2023-09-12T14:56:22.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.764+0000 7f85657fa640 1 -- 172.21.15.131:0/796434842 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85600ad8b0 con 0x7f85680d1680 2023-09-12T14:56:22.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.764+0000 7f856ee3d640 1 -- 172.21.15.131:0/796434842 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85681520b0 con 0x7f85680d1680 2023-09-12T14:56:22.776 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.764+0000 7f856ee3d640 1 -- 172.21.15.131:0/796434842 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8568073580 con 0x7f85680d1680 2023-09-12T14:56:22.776 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.764+0000 7f85657fa640 1 -- 172.21.15.131:0/796434842 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f85600b4040 con 0x7f85680d1680 2023-09-12T14:56:22.776 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.764+0000 7f85657fa640 1 --2- 172.21.15.131:0/796434842 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8554065b20 0x7f8554067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:22.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.764+0000 7f856cbd9640 1 --2- 172.21.15.131:0/796434842 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8554065b20 0x7f8554067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:22.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.764+0000 7f85657fa640 1 -- 172.21.15.131:0/796434842 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f856011d8f0 con 0x7f85680d1680 2023-09-12T14:56:22.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.764+0000 7f856cbd9640 1 --2- 172.21.15.131:0/796434842 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8554065b20 0x7f8554067fe0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f855c00ba40 tx=0x7f855c005cd0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:22.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:22.768+0000 7f85657fa640 1 -- 172.21.15.131:0/796434842 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f85600eb540 con 0x7f85680d1680 2023-09-12T14:56:23.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:23.000+0000 7f856ee3d640 1 -- 172.21.15.131:0/796434842 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values"]} v 0) v1 -- 0x7f85680c9af0 con 0x7f85680d1680 2023-09-12T14:56:23.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:23.300+0000 7f85657fa640 1 -- 172.21.15.131:0/796434842 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values"]}]=0 v101) v1 ==== 160+0+0 (secure 0 0 0) 0x7f85600b2020 con 0x7f85680d1680 2023-09-12T14:56:23.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:23.304+0000 7f856ee3d640 1 -- 172.21.15.131:0/796434842 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8554065b20 msgr2=0x7f8554067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:23.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:23.304+0000 7f856ee3d640 1 --2- 172.21.15.131:0/796434842 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8554065b20 0x7f8554067fe0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f855c00ba40 tx=0x7f855c005cd0 comp rx=0 tx=0).stop 2023-09-12T14:56:23.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:23.304+0000 7f856ee3d640 1 -- 172.21.15.131:0/796434842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85680d1680 msgr2=0x7f85680cd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:23.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:23.304+0000 7f856ee3d640 1 --2- 172.21.15.131:0/796434842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85680d1680 0x7f85680cd130 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8560009bd0 tx=0x7f8560003c40 comp rx=0 tx=0).stop 2023-09-12T14:56:23.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:23.304+0000 7f856ee3d640 1 -- 172.21.15.131:0/796434842 shutdown_connections 2023-09-12T14:56:23.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:23.304+0000 7f856ee3d640 1 --2- 172.21.15.131:0/796434842 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8554065b20 0x7f8554067fe0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:23.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:23.304+0000 7f856ee3d640 1 --2- 172.21.15.131:0/796434842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85680cf2d0 0x7f856815ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:23.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:23.304+0000 7f856ee3d640 1 --2- 172.21.15.131:0/796434842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85680d1680 0x7f85680cd130 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:23.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:23.304+0000 7f856ee3d640 1 --2- 172.21.15.131:0/796434842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85680b1760 0x7f8568151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:23.310 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:23.304+0000 7f856ee3d640 1 -- 172.21.15.131:0/796434842 >> 172.21.15.131:0/796434842 conn(0x7f856809a200 msgr2=0x7f85680cbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:23.310 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:23.304+0000 7f856ee3d640 1 -- 172.21.15.131:0/796434842 shutdown_connections 2023-09-12T14:56:23.310 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:23.304+0000 7f856ee3d640 1 -- 172.21.15.131:0/796434842 wait complete. 2023-09-12T14:56:23.328 INFO:tasks.cephfs_test_runner:test: ... ERROR 2023-09-12T14:56:23.329 INFO:tasks.cephfs_test_runner: 2023-09-12T14:56:23.329 INFO:tasks.cephfs_test_runner:====================================================================== 2023-09-12T14:56:23.330 INFO:tasks.cephfs_test_runner:ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-09-12T14:56:23.330 INFO:tasks.cephfs_test_runner:test: 2023-09-12T14:56:23.331 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-09-12T14:56:23.331 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-09-12T14:56:23.332 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks/cephfs/test_quota.py", line 157, in test_disable_enable_human_readable_quota_values 2023-09-12T14:56:23.332 INFO:tasks.cephfs_test_runner: self.mount_a.setfattr("./subdir", "ceph.quota.max_bytes", "1K") 2023-09-12T14:56:23.333 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks/cephfs/mount.py", line 1566, in setfattr 2023-09-12T14:56:23.333 INFO:tasks.cephfs_test_runner: self.run_shell(**kwargs) 2023-09-12T14:56:23.334 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks/cephfs/mount.py", line 780, in run_shell 2023-09-12T14:56:23.334 INFO:tasks.cephfs_test_runner: return self.client_remote.run(args=args, cwd=cwd, timeout=timeout, 2023-09-12T14:56:23.335 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run 2023-09-12T14:56:23.335 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-09-12T14:56:23.336 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run 2023-09-12T14:56:23.336 INFO:tasks.cephfs_test_runner: r.wait() 2023-09-12T14:56:23.337 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait 2023-09-12T14:56:23.337 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-09-12T14:56:23.338 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-09-12T14:56:23.338 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-09-12T14:56:23.339 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi131 with status 1: '(cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quota.max_bytes -v 1K ./subdir)' 2023-09-12T14:56:23.339 INFO:tasks.cephfs_test_runner: 2023-09-12T14:56:23.340 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-09-12T14:56:23.340 INFO:tasks.cephfs_test_runner:Ran 1 test in 55.109s 2023-09-12T14:56:23.341 INFO:tasks.cephfs_test_runner: 2023-09-12T14:56:23.341 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-09-12T14:56:23.342 INFO:tasks.cephfs_test_runner: 2023-09-12T14:56:23.342 INFO:tasks.cephfs_test_runner:====================================================================== 2023-09-12T14:56:23.343 INFO:tasks.cephfs_test_runner:ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-09-12T14:56:23.343 INFO:tasks.cephfs_test_runner:test: 2023-09-12T14:56:23.344 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-09-12T14:56:23.344 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-09-12T14:56:23.345 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks/cephfs/test_quota.py", line 157, in test_disable_enable_human_readable_quota_values 2023-09-12T14:56:23.345 INFO:tasks.cephfs_test_runner: self.mount_a.setfattr("./subdir", "ceph.quota.max_bytes", "1K") 2023-09-12T14:56:23.346 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks/cephfs/mount.py", line 1566, in setfattr 2023-09-12T14:56:23.346 INFO:tasks.cephfs_test_runner: self.run_shell(**kwargs) 2023-09-12T14:56:23.347 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks/cephfs/mount.py", line 780, in run_shell 2023-09-12T14:56:23.347 INFO:tasks.cephfs_test_runner: return self.client_remote.run(args=args, cwd=cwd, timeout=timeout, 2023-09-12T14:56:23.347 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run 2023-09-12T14:56:23.348 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-09-12T14:56:23.348 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run 2023-09-12T14:56:23.349 INFO:tasks.cephfs_test_runner: r.wait() 2023-09-12T14:56:23.349 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait 2023-09-12T14:56:23.350 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-09-12T14:56:23.351 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-09-12T14:56:23.351 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-09-12T14:56:23.352 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi131 with status 1: '(cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quota.max_bytes -v 1K ./subdir)' 2023-09-12T14:56:23.352 INFO:tasks.cephfs_test_runner: 2023-09-12T14:56:23.353 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-09-12T14:56:23.417 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=7f383fe11333408a9de717dc11637562 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-09-12T14:56:23.419 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-09-12T14:56:23.430 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-09-12T14:56:23.452 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-09-12T14:56:23.453 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:56:23.453 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-12T14:56:23.467 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18514 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-09-12T14:56:23.468 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-09-12T14:56:23.468 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-12T14:56:23.514 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18514 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-09-12T14:56:23.515 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:56:23.515 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-12T14:56:23.521 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19313 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-09-12T14:56:23.522 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-09-12T14:56:23.522 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-12T14:56:23.571 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19313 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-09-12T14:56:23.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:23.572 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2023-09-12T14:56:23.578 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-09-12T14:56:23.579 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-12T14:56:23.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:23.579 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-12T14:56:23.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:23.636 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-12T14:56:23.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:23.699 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link delete ceph-brx 2023-09-12T14:56:23.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:23.872 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2023-09-12T14:56:23.878 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-09-12T14:56:23.879 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-09-12T14:56:23.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:23.879 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-09-12T14:56:23.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:23.936 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-09-12T14:56:24.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-12T14:56:24.003 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link delete ceph-brx 2023-09-12T14:56:24.184 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-09-12T14:56:24.193 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-09-12T14:56:24.193 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-12T14:56:24.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e9dca0640 1 -- 172.21.15.131:0/1497798189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e9806c060 msgr2=0x7f5e981626d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:24.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e9dca0640 1 --2- 172.21.15.131:0/1497798189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e9806c060 0x7f5e981626d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c009840 tx=0x7f5e8c01c740 comp rx=0 tx=0).stop 2023-09-12T14:56:24.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e9dca0640 1 -- 172.21.15.131:0/1497798189 shutdown_connections 2023-09-12T14:56:24.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e9dca0640 1 --2- 172.21.15.131:0/1497798189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e9806c060 0x7f5e981626d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:24.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e9dca0640 1 -- 172.21.15.131:0/1497798189 >> 172.21.15.131:0/1497798189 conn(0x7f5e98069840 msgr2=0x7f5e980cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:24.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e9dca0640 1 -- 172.21.15.131:0/1497798189 shutdown_connections 2023-09-12T14:56:24.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e9dca0640 1 -- 172.21.15.131:0/1497798189 wait complete. 2023-09-12T14:56:24.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e9dca0640 1 Processor -- start 2023-09-12T14:56:24.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e9dca0640 1 -- start start 2023-09-12T14:56:24.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e9dca0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e9806c060 0x7f5e9814ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:24.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e9dca0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e980ab730 0x7f5e98147630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:24.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e9dca0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e98158180 0x7f5e98156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:24.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e9dca0640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e98148040 con 0x7f5e98158180 2023-09-12T14:56:24.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e9dca0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e98148320 con 0x7f5e9806c060 2023-09-12T14:56:24.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e9dca0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e98148600 con 0x7f5e980ab730 2023-09-12T14:56:24.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e97fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e98158180 0x7f5e98156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:24.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e97fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e98158180 0x7f5e98156b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44992/0 (socket says 172.21.15.131:44992) 2023-09-12T14:56:24.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e97fff640 1 -- 172.21.15.131:0/1966378420 learned_addr learned my addr 172.21.15.131:0/1966378420 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:24.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e977fe640 1 --2- 172.21.15.131:0/1966378420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e9806c060 0x7f5e9814ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:24.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e96ffd640 1 --2- 172.21.15.131:0/1966378420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e980ab730 0x7f5e98147630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:24.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e97fff640 1 -- 172.21.15.131:0/1966378420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e980ab730 msgr2=0x7f5e98147630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:24.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e97fff640 1 --2- 172.21.15.131:0/1966378420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e980ab730 0x7f5e98147630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:24.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e97fff640 1 -- 172.21.15.131:0/1966378420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e9806c060 msgr2=0x7f5e9814ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:24.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e97fff640 1 --2- 172.21.15.131:0/1966378420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e9806c060 0x7f5e9814ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:24.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e97fff640 1 -- 172.21.15.131:0/1966378420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8c0094a0 con 0x7f5e98158180 2023-09-12T14:56:24.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e977fe640 1 --2- 172.21.15.131:0/1966378420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e9806c060 0x7f5e9814ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:56:24.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e97fff640 1 --2- 172.21.15.131:0/1966378420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e98158180 0x7f5e98156b70 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c019350 tx=0x7f5e8c0a57e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:24.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e7ffff640 1 -- 172.21.15.131:0/1966378420 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c0042c0 con 0x7f5e98158180 2023-09-12T14:56:24.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e9dca0640 1 -- 172.21.15.131:0/1966378420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e981488e0 con 0x7f5e98158180 2023-09-12T14:56:24.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e7ffff640 1 -- 172.21.15.131:0/1966378420 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5e8c004510 con 0x7f5e98158180 2023-09-12T14:56:24.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e7ffff640 1 -- 172.21.15.131:0/1966378420 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c003860 con 0x7f5e98158180 2023-09-12T14:56:24.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.324+0000 7f5e9dca0640 1 -- 172.21.15.131:0/1966378420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e9814f1a0 con 0x7f5e98158180 2023-09-12T14:56:24.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.328+0000 7f5e9dca0640 1 -- 172.21.15.131:0/1966378420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e98073580 con 0x7f5e98158180 2023-09-12T14:56:24.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.328+0000 7f5e7ffff640 1 -- 172.21.15.131:0/1966378420 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f5e8c0b4040 con 0x7f5e98158180 2023-09-12T14:56:24.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.328+0000 7f5e7ffff640 1 --2- 172.21.15.131:0/1966378420 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5e84065b50 0x7f5e84068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:24.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.328+0000 7f5e7ffff640 1 -- 172.21.15.131:0/1966378420 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5e8c11da00 con 0x7f5e98158180 2023-09-12T14:56:24.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.328+0000 7f5e977fe640 1 --2- 172.21.15.131:0/1966378420 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5e84065b50 0x7f5e84068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:24.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.328+0000 7f5e977fe640 1 --2- 172.21.15.131:0/1966378420 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5e84065b50 0x7f5e84068010 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5e900011f0 tx=0x7f5e90009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:24.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.332+0000 7f5e7ffff640 1 -- 172.21.15.131:0/1966378420 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f5e8c0eb660 con 0x7f5e98158180 2023-09-12T14:56:24.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.484+0000 7f5e9dca0640 1 -- 172.21.15.131:0/1966378420 --> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5e98148d90 con 0x7f5e84065b50 2023-09-12T14:56:24.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.488+0000 7f5e7ffff640 1 -- 172.21.15.131:0/1966378420 <== mgr.4200 v2:172.21.15.131:6832/2298103651 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49392 (secure 0 0 0) 0x7f5e98148d90 con 0x7f5e84065b50 2023-09-12T14:56:24.491 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:56:24.491 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-09-12T14:56:24.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.488+0000 7f5e9dca0640 1 -- 172.21.15.131:0/1966378420 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5e84065b50 msgr2=0x7f5e84068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:24.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.488+0000 7f5e9dca0640 1 --2- 172.21.15.131:0/1966378420 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5e84065b50 0x7f5e84068010 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5e900011f0 tx=0x7f5e90009040 comp rx=0 tx=0).stop 2023-09-12T14:56:24.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.488+0000 7f5e9dca0640 1 -- 172.21.15.131:0/1966378420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e98158180 msgr2=0x7f5e98156b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:24.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.488+0000 7f5e9dca0640 1 --2- 172.21.15.131:0/1966378420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e98158180 0x7f5e98156b70 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c019350 tx=0x7f5e8c0a57e0 comp rx=0 tx=0).stop 2023-09-12T14:56:24.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.488+0000 7f5e9dca0640 1 -- 172.21.15.131:0/1966378420 shutdown_connections 2023-09-12T14:56:24.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.488+0000 7f5e9dca0640 1 --2- 172.21.15.131:0/1966378420 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5e84065b50 0x7f5e84068010 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:24.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.488+0000 7f5e9dca0640 1 --2- 172.21.15.131:0/1966378420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e980ab730 0x7f5e98147630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:24.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.488+0000 7f5e9dca0640 1 --2- 172.21.15.131:0/1966378420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e98158180 0x7f5e98156b70 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:24.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.488+0000 7f5e9dca0640 1 --2- 172.21.15.131:0/1966378420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e9806c060 0x7f5e9814ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:24.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.488+0000 7f5e9dca0640 1 -- 172.21.15.131:0/1966378420 >> 172.21.15.131:0/1966378420 conn(0x7f5e98069840 msgr2=0x7f5e9806f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:24.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.488+0000 7f5e9dca0640 1 -- 172.21.15.131:0/1966378420 shutdown_connections 2023-09-12T14:56:24.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.488+0000 7f5e9dca0640 1 -- 172.21.15.131:0/1966378420 wait complete. 2023-09-12T14:56:24.509 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2023-09-12T14:56:23.603035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224532,"kb_used_data":4212,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750188,"statfs":{"total":773094113280,"available":772864192512,"internally_reserved":0,"allocated":4313088,"data_stored":2404461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":225496632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6276,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-50,"num_write_kb":-21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001901"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":177,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.188683+0000","last_change":"2023-09-12T14:54:46.965446+0000","last_active":"2023-09-12T14:55:59.188683+0000","last_peered":"2023-09-12T14:55:59.188683+0000","last_clean":"2023-09-12T14:55:59.188683+0000","last_became_active":"2023-09-12T14:54:46.963236+0000","last_became_peered":"2023-09-12T14:54:46.963236+0000","last_unstale":"2023-09-12T14:55:59.188683+0000","last_undegraded":"2023-09-12T14:55:59.188683+0000","last_fullsized":"2023-09-12T14:55:59.188683+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:45.804232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:45.804232+0000","last_clean_scrub_stamp":"2023-09-12T14:54:45.804232+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:24:45.670513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":983040,"data_stored":743467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.70799999999999996}]},{"osd":1,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.89200000000000002}]},{"osd":2,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.089}]},{"osd":3,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.98999999999999999}]},{"osd":4,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.123}]},{"osd":5,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.1659999999999999}]},{"osd":6,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.198}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":153625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":1.1279999999999999}]},{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":1.306}]},{"osd":2,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.72999999999999998}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.2609999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.89000000000000001}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":1.175}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.98599999999999999}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.22},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":1.345}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.84199999999999997}]},{"osd":1,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.749},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.98499999999999999}]},{"osd":2,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.63500000000000001}]},{"osd":3,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":1.0569999999999999}]},{"osd":4,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.878},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.097}]},{"osd":6,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.516,"5min":2.516,"15min":2.516},"last":1.137},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.94799999999999995}]},{"osd":7,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":1.401}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":153113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.497},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.753}]},{"osd":1,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.80800000000000005}]},{"osd":2,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":1.1200000000000001},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":1.0329999999999999}]},{"osd":3,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.85599999999999998}]},{"osd":5,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.64200000000000002}]},{"osd":6,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":1.167},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":1.097}]},{"osd":7,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":1.002},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":1.0629999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.39300000000000002}]},{"osd":1,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.38},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.29999999999999999}]},{"osd":2,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.36499999999999999}]},{"osd":4,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.79200000000000004}]},{"osd":5,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.73199999999999998}]},{"osd":6,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.63900000000000001}]},{"osd":7,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.86699999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":146998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.33300000000000002}]},{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.45300000000000001}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.51700000000000002}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.875}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.82599999999999996}]},{"osd":6,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.1989999999999998,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":0.78000000000000003}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.70599999999999996},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.90800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.57999999999999996}]},{"osd":2,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.98899999999999999}]},{"osd":3,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":1.1339999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.94299999999999995}]},{"osd":4,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.29}]},{"osd":5,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":1.222}]},{"osd":6,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.90100000000000002}]},{"osd":7,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":1.196},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.2609999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":974848,"data_stored":742358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.099}]},{"osd":2,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.91400000000000003}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.73299999999999998}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.0289999999999999}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.504}]},{"osd":6,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.169}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.1379999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-12T14:56:24.512 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-12T14:56:24.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.632+0000 7f5aee6a3640 1 -- 172.21.15.131:0/1303460909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ad00062a0 msgr2=0x7f5ad000caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:24.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.632+0000 7f5aee6a3640 1 --2- 172.21.15.131:0/1303460909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ad00062a0 0x7f5ad000caa0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009eb0 tx=0x7f5ae001d5d0 comp rx=0 tx=0).stop 2023-09-12T14:56:24.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.632+0000 7f5aee6a3640 1 -- 172.21.15.131:0/1303460909 shutdown_connections 2023-09-12T14:56:24.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.632+0000 7f5aee6a3640 1 --2- 172.21.15.131:0/1303460909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ad00062a0 0x7f5ad000caa0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:24.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.632+0000 7f5aee6a3640 1 -- 172.21.15.131:0/1303460909 >> 172.21.15.131:0/1303460909 conn(0x7f5ae8069840 msgr2=0x7f5ae80cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:24.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.632+0000 7f5aee6a3640 1 -- 172.21.15.131:0/1303460909 shutdown_connections 2023-09-12T14:56:24.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.632+0000 7f5aee6a3640 1 -- 172.21.15.131:0/1303460909 wait complete. 2023-09-12T14:56:24.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.636+0000 7f5aee6a3640 1 Processor -- start 2023-09-12T14:56:24.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.636+0000 7f5aee6a3640 1 -- start start 2023-09-12T14:56:24.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.636+0000 7f5aee6a3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ad00062a0 0x7f5ae81443a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:24.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.636+0000 7f5aee6a3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8159960 0x7f5ae8159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:24.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.636+0000 7f5aee6a3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ae8158360 0x7f5ae81587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:24.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.636+0000 7f5aee6a3640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae81f0160 con 0x7f5ae8159960 2023-09-12T14:56:24.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.636+0000 7f5aee6a3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae815a7e0 con 0x7f5ad00062a0 2023-09-12T14:56:24.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.636+0000 7f5aee6a3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae815a300 con 0x7f5ae8158360 2023-09-12T14:56:24.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.636+0000 7f5ae77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8159960 0x7f5ae8159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:24.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.636+0000 7f5ae77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8159960 0x7f5ae8159dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45012/0 (socket says 172.21.15.131:45012) 2023-09-12T14:56:24.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.636+0000 7f5ae77fe640 1 -- 172.21.15.131:0/1245420161 learned_addr learned my addr 172.21.15.131:0/1245420161 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:24.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.636+0000 7f5aecc40640 1 --2- 172.21.15.131:0/1245420161 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ae8158360 0x7f5ae81587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:24.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.636+0000 7f5ae7fff640 1 --2- 172.21.15.131:0/1245420161 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ad00062a0 0x7f5ae81443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:24.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.636+0000 7f5ae77fe640 1 -- 172.21.15.131:0/1245420161 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ae8158360 msgr2=0x7f5ae81587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:24.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.636+0000 7f5ae77fe640 1 --2- 172.21.15.131:0/1245420161 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ae8158360 0x7f5ae81587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:24.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.636+0000 7f5ae77fe640 1 -- 172.21.15.131:0/1245420161 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ad00062a0 msgr2=0x7f5ae81443a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:24.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.636+0000 7f5ae77fe640 1 --2- 172.21.15.131:0/1245420161 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ad00062a0 0x7f5ae81443a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:24.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.636+0000 7f5ae77fe640 1 -- 172.21.15.131:0/1245420161 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae8158d20 con 0x7f5ae8159960 2023-09-12T14:56:24.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.636+0000 7f5ae7fff640 1 --2- 172.21.15.131:0/1245420161 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ad00062a0 0x7f5ae81443a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:24.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.636+0000 7f5aecc40640 1 --2- 172.21.15.131:0/1245420161 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ae8158360 0x7f5ae81587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:24.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.636+0000 7f5ae77fe640 1 --2- 172.21.15.131:0/1245420161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8159960 0x7f5ae8159dc0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5adc00d500 tx=0x7f5adc00d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:24.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.636+0000 7f5ae4ff9640 1 -- 172.21.15.131:0/1245420161 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc012630 con 0x7f5ae8159960 2023-09-12T14:56:24.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.636+0000 7f5ae4ff9640 1 -- 172.21.15.131:0/1245420161 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5adc012790 con 0x7f5ae8159960 2023-09-12T14:56:24.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.636+0000 7f5aee6a3640 1 -- 172.21.15.131:0/1245420161 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae8158f70 con 0x7f5ae8159960 2023-09-12T14:56:24.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.636+0000 7f5ae4ff9640 1 -- 172.21.15.131:0/1245420161 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc012ba0 con 0x7f5ae8159960 2023-09-12T14:56:24.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.636+0000 7f5aee6a3640 1 -- 172.21.15.131:0/1245420161 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae81f0340 con 0x7f5ae8159960 2023-09-12T14:56:24.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.636+0000 7f5aee6a3640 1 -- 172.21.15.131:0/1245420161 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae8159490 con 0x7f5ae8159960 2023-09-12T14:56:24.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.640+0000 7f5ae4ff9640 1 -- 172.21.15.131:0/1245420161 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f5adc01b400 con 0x7f5ae8159960 2023-09-12T14:56:24.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.640+0000 7f5ae4ff9640 1 --2- 172.21.15.131:0/1245420161 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5ad4065b20 0x7f5ad4067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:24.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.640+0000 7f5ae7fff640 1 --2- 172.21.15.131:0/1245420161 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5ad4065b20 0x7f5ad4067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:24.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.640+0000 7f5ae4ff9640 1 -- 172.21.15.131:0/1245420161 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5adc083190 con 0x7f5ae8159960 2023-09-12T14:56:24.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.640+0000 7f5ae7fff640 1 --2- 172.21.15.131:0/1245420161 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5ad4065b20 0x7f5ad4067fe0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5ad80017b0 tx=0x7f5ad8001460 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:24.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.644+0000 7f5ae4ff9640 1 -- 172.21.15.131:0/1245420161 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f5ae8159490 con 0x7f5ae8159960 2023-09-12T14:56:24.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.792+0000 7f5aee6a3640 1 -- 172.21.15.131:0/1245420161 --> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5ae8159490 con 0x7f5ad4065b20 2023-09-12T14:56:24.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.800+0000 7f5ae4ff9640 1 -- 172.21.15.131:0/1245420161 <== mgr.4200 v2:172.21.15.131:6832/2298103651 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49392 (secure 0 0 0) 0x7f5ae8159490 con 0x7f5ad4065b20 2023-09-12T14:56:24.803 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:56:24.803 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-09-12T14:56:24.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.800+0000 7f5aee6a3640 1 -- 172.21.15.131:0/1245420161 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5ad4065b20 msgr2=0x7f5ad4067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:24.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.800+0000 7f5aee6a3640 1 --2- 172.21.15.131:0/1245420161 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5ad4065b20 0x7f5ad4067fe0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5ad80017b0 tx=0x7f5ad8001460 comp rx=0 tx=0).stop 2023-09-12T14:56:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.800+0000 7f5aee6a3640 1 -- 172.21.15.131:0/1245420161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8159960 msgr2=0x7f5ae8159dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.800+0000 7f5aee6a3640 1 --2- 172.21.15.131:0/1245420161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8159960 0x7f5ae8159dc0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5adc00d500 tx=0x7f5adc00d9c0 comp rx=0 tx=0).stop 2023-09-12T14:56:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.800+0000 7f5aee6a3640 1 -- 172.21.15.131:0/1245420161 shutdown_connections 2023-09-12T14:56:24.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.800+0000 7f5aee6a3640 1 --2- 172.21.15.131:0/1245420161 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5ad4065b20 0x7f5ad4067fe0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:24.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.800+0000 7f5aee6a3640 1 --2- 172.21.15.131:0/1245420161 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ae8158360 0x7f5ae81587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:24.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.800+0000 7f5aee6a3640 1 --2- 172.21.15.131:0/1245420161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8159960 0x7f5ae8159dc0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:24.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.800+0000 7f5aee6a3640 1 --2- 172.21.15.131:0/1245420161 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ad00062a0 0x7f5ae81443a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:24.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.800+0000 7f5aee6a3640 1 -- 172.21.15.131:0/1245420161 >> 172.21.15.131:0/1245420161 conn(0x7f5ae8069840 msgr2=0x7f5ae80adc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:24.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.800+0000 7f5aee6a3640 1 -- 172.21.15.131:0/1245420161 shutdown_connections 2023-09-12T14:56:24.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.800+0000 7f5aee6a3640 1 -- 172.21.15.131:0/1245420161 wait complete. 2023-09-12T14:56:24.822 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2023-09-12T14:56:23.603035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224532,"kb_used_data":4212,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750188,"statfs":{"total":773094113280,"available":772864192512,"internally_reserved":0,"allocated":4313088,"data_stored":2404461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":225496632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6276,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-50,"num_write_kb":-21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001901"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":177,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.188683+0000","last_change":"2023-09-12T14:54:46.965446+0000","last_active":"2023-09-12T14:55:59.188683+0000","last_peered":"2023-09-12T14:55:59.188683+0000","last_clean":"2023-09-12T14:55:59.188683+0000","last_became_active":"2023-09-12T14:54:46.963236+0000","last_became_peered":"2023-09-12T14:54:46.963236+0000","last_unstale":"2023-09-12T14:55:59.188683+0000","last_undegraded":"2023-09-12T14:55:59.188683+0000","last_fullsized":"2023-09-12T14:55:59.188683+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:45.804232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:45.804232+0000","last_clean_scrub_stamp":"2023-09-12T14:54:45.804232+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:24:45.670513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":983040,"data_stored":743467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.70799999999999996}]},{"osd":1,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.89200000000000002}]},{"osd":2,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.089}]},{"osd":3,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.98999999999999999}]},{"osd":4,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.123}]},{"osd":5,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.1659999999999999}]},{"osd":6,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.198}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":153625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":1.1279999999999999}]},{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":1.306}]},{"osd":2,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.72999999999999998}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.2609999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.89000000000000001}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":1.175}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.98599999999999999}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.22},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":1.345}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.84199999999999997}]},{"osd":1,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.749},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.98499999999999999}]},{"osd":2,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.63500000000000001}]},{"osd":3,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":1.0569999999999999}]},{"osd":4,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.878},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.097}]},{"osd":6,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.516,"5min":2.516,"15min":2.516},"last":1.137},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.94799999999999995}]},{"osd":7,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":1.401}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":153113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.497},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.753}]},{"osd":1,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.80800000000000005}]},{"osd":2,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":1.1200000000000001},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":1.0329999999999999}]},{"osd":3,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.85599999999999998}]},{"osd":5,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.64200000000000002}]},{"osd":6,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":1.167},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":1.097}]},{"osd":7,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":1.002},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":1.0629999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.39300000000000002}]},{"osd":1,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.38},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.29999999999999999}]},{"osd":2,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.36499999999999999}]},{"osd":4,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.79200000000000004}]},{"osd":5,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.73199999999999998}]},{"osd":6,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.63900000000000001}]},{"osd":7,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.86699999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":146998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.33300000000000002}]},{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.45300000000000001}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.51700000000000002}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.875}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.82599999999999996}]},{"osd":6,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.1989999999999998,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":0.78000000000000003}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.70599999999999996},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.90800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.57999999999999996}]},{"osd":2,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.98899999999999999}]},{"osd":3,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":1.1339999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.94299999999999995}]},{"osd":4,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.29}]},{"osd":5,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":1.222}]},{"osd":6,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.90100000000000002}]},{"osd":7,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":1.196},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.2609999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":974848,"data_stored":742358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.099}]},{"osd":2,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.91400000000000003}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.73299999999999998}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.0289999999999999}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.504}]},{"osd":6,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.169}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.1379999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-12T14:56:24.824 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-09-12T14:56:24.824 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-12T14:56:24.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.948+0000 7fef5e172640 1 -- 172.21.15.131:0/1282447205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef5806b050 msgr2=0x7fef5806b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:24.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.948+0000 7fef5e172640 1 --2- 172.21.15.131:0/1282447205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef5806b050 0x7fef5806b460 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fef5806eea0 tx=0x7fef5001cb30 comp rx=0 tx=0).stop 2023-09-12T14:56:24.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.948+0000 7fef5e172640 1 -- 172.21.15.131:0/1282447205 shutdown_connections 2023-09-12T14:56:24.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.948+0000 7fef5e172640 1 --2- 172.21.15.131:0/1282447205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef5806b050 0x7fef5806b460 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:24.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.948+0000 7fef5e172640 1 --2- 172.21.15.131:0/1282447205 >> v2:172.21.15.187:3300/0 conn(0x7fef580ab730 0x7fef580abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:24.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.948+0000 7fef5e172640 1 -- 172.21.15.131:0/1282447205 >> 172.21.15.131:0/1282447205 conn(0x7fef58069840 msgr2=0x7fef580cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:24.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.948+0000 7fef5e172640 1 -- 172.21.15.131:0/1282447205 shutdown_connections 2023-09-12T14:56:24.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.948+0000 7fef5e172640 1 -- 172.21.15.131:0/1282447205 wait complete. 2023-09-12T14:56:24.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.948+0000 7fef5e172640 1 Processor -- start 2023-09-12T14:56:24.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.948+0000 7fef5e172640 1 -- start start 2023-09-12T14:56:24.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.948+0000 7fef5e172640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef5806b050 0x7fef581401e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:24.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.948+0000 7fef5e172640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef580ab730 0x7fef58155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:24.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.948+0000 7fef5e172640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef58158180 0x7fef58156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:24.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.948+0000 7fef5e172640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef581541c0 con 0x7fef58158180 2023-09-12T14:56:24.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.948+0000 7fef5e172640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef581544a0 con 0x7fef5806b050 2023-09-12T14:56:24.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.948+0000 7fef5e172640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef58154780 con 0x7fef580ab730 2023-09-12T14:56:24.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.948+0000 7fef57fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef58158180 0x7fef58156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:24.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.948+0000 7fef57fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef58158180 0x7fef58156110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45040/0 (socket says 172.21.15.131:45040) 2023-09-12T14:56:24.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.948+0000 7fef57fff640 1 -- 172.21.15.131:0/1523157846 learned_addr learned my addr 172.21.15.131:0/1523157846 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:24.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.948+0000 7fef577fe640 1 --2- 172.21.15.131:0/1523157846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef5806b050 0x7fef581401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:24.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.948+0000 7fef56ffd640 1 --2- 172.21.15.131:0/1523157846 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef580ab730 0x7fef58155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:24.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.948+0000 7fef57fff640 1 -- 172.21.15.131:0/1523157846 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef580ab730 msgr2=0x7fef58155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:24.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.948+0000 7fef57fff640 1 --2- 172.21.15.131:0/1523157846 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef580ab730 0x7fef58155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:24.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.948+0000 7fef57fff640 1 -- 172.21.15.131:0/1523157846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef5806b050 msgr2=0x7fef581401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:24.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.948+0000 7fef57fff640 1 --2- 172.21.15.131:0/1523157846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef5806b050 0x7fef581401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:24.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.948+0000 7fef57fff640 1 -- 172.21.15.131:0/1523157846 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef50009820 con 0x7fef58158180 2023-09-12T14:56:24.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.952+0000 7fef57fff640 1 --2- 172.21.15.131:0/1523157846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef58158180 0x7fef58156110 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fef50009b10 tx=0x7fef500a5a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:24.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.952+0000 7fef3ffff640 1 -- 172.21.15.131:0/1523157846 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef50003b80 con 0x7fef58158180 2023-09-12T14:56:24.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.952+0000 7fef3ffff640 1 -- 172.21.15.131:0/1523157846 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fef50003ce0 con 0x7fef58158180 2023-09-12T14:56:24.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.952+0000 7fef5e172640 1 -- 172.21.15.131:0/1523157846 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef58154a60 con 0x7fef58158180 2023-09-12T14:56:24.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.952+0000 7fef3ffff640 1 -- 172.21.15.131:0/1523157846 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef50004290 con 0x7fef58158180 2023-09-12T14:56:24.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.952+0000 7fef5e172640 1 -- 172.21.15.131:0/1523157846 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef58140720 con 0x7fef58158180 2023-09-12T14:56:24.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.952+0000 7fef5e172640 1 -- 172.21.15.131:0/1523157846 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef58073580 con 0x7fef58158180 2023-09-12T14:56:24.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.952+0000 7fef3ffff640 1 -- 172.21.15.131:0/1523157846 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fef500b4040 con 0x7fef58158180 2023-09-12T14:56:24.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.952+0000 7fef3ffff640 1 --2- 172.21.15.131:0/1523157846 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fef44065b20 0x7fef44067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:24.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.952+0000 7fef3ffff640 1 -- 172.21.15.131:0/1523157846 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fef5011dc20 con 0x7fef58158180 2023-09-12T14:56:24.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.952+0000 7fef577fe640 1 --2- 172.21.15.131:0/1523157846 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fef44065b20 0x7fef44067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:24.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.952+0000 7fef577fe640 1 --2- 172.21.15.131:0/1523157846 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fef44065b20 0x7fef44067fe0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fef4c009b70 tx=0x7fef4c0091e0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:24.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:24.956+0000 7fef3ffff640 1 -- 172.21.15.131:0/1523157846 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fef500eb900 con 0x7fef58158180 2023-09-12T14:56:25.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.096+0000 7fef5e172640 1 -- 172.21.15.131:0/1523157846 --> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fef58154f10 con 0x7fef44065b20 2023-09-12T14:56:25.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.100+0000 7fef3ffff640 1 -- 172.21.15.131:0/1523157846 <== mgr.4200 v2:172.21.15.131:6832/2298103651 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49392 (secure 0 0 0) 0x7fef58154f10 con 0x7fef44065b20 2023-09-12T14:56:25.104 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:56:25.105 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-09-12T14:56:25.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.104+0000 7fef5e172640 1 -- 172.21.15.131:0/1523157846 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fef44065b20 msgr2=0x7fef44067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:25.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.104+0000 7fef5e172640 1 --2- 172.21.15.131:0/1523157846 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fef44065b20 0x7fef44067fe0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fef4c009b70 tx=0x7fef4c0091e0 comp rx=0 tx=0).stop 2023-09-12T14:56:25.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.104+0000 7fef5e172640 1 -- 172.21.15.131:0/1523157846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef58158180 msgr2=0x7fef58156110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:25.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.104+0000 7fef5e172640 1 --2- 172.21.15.131:0/1523157846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef58158180 0x7fef58156110 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fef50009b10 tx=0x7fef500a5a80 comp rx=0 tx=0).stop 2023-09-12T14:56:25.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.104+0000 7fef5e172640 1 -- 172.21.15.131:0/1523157846 shutdown_connections 2023-09-12T14:56:25.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.104+0000 7fef5e172640 1 --2- 172.21.15.131:0/1523157846 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fef44065b20 0x7fef44067fe0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:25.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.104+0000 7fef5e172640 1 --2- 172.21.15.131:0/1523157846 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef580ab730 0x7fef58155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:25.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.104+0000 7fef5e172640 1 --2- 172.21.15.131:0/1523157846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef58158180 0x7fef58156110 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:25.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.104+0000 7fef5e172640 1 --2- 172.21.15.131:0/1523157846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef5806b050 0x7fef581401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:25.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.104+0000 7fef5e172640 1 -- 172.21.15.131:0/1523157846 >> 172.21.15.131:0/1523157846 conn(0x7fef58069840 msgr2=0x7fef580c9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:25.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.104+0000 7fef5e172640 1 -- 172.21.15.131:0/1523157846 shutdown_connections 2023-09-12T14:56:25.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.104+0000 7fef5e172640 1 -- 172.21.15.131:0/1523157846 wait complete. 2023-09-12T14:56:25.121 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2023-09-12T14:56:23.603035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224532,"kb_used_data":4212,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750188,"statfs":{"total":773094113280,"available":772864192512,"internally_reserved":0,"allocated":4313088,"data_stored":2404461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":225496632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6276,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-50,"num_write_kb":-21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001901"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":177,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-09-12T14:55:59.188683+0000","last_change":"2023-09-12T14:54:46.965446+0000","last_active":"2023-09-12T14:55:59.188683+0000","last_peered":"2023-09-12T14:55:59.188683+0000","last_clean":"2023-09-12T14:55:59.188683+0000","last_became_active":"2023-09-12T14:54:46.963236+0000","last_became_peered":"2023-09-12T14:54:46.963236+0000","last_unstale":"2023-09-12T14:55:59.188683+0000","last_undegraded":"2023-09-12T14:55:59.188683+0000","last_fullsized":"2023-09-12T14:55:59.188683+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:45.804232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:45.804232+0000","last_clean_scrub_stamp":"2023-09-12T14:54:45.804232+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:24:45.670513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":983040,"data_stored":743467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.70799999999999996}]},{"osd":1,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.89200000000000002}]},{"osd":2,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.089}]},{"osd":3,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.98999999999999999}]},{"osd":4,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.123}]},{"osd":5,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.1659999999999999}]},{"osd":6,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.198}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":153625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":1.1279999999999999}]},{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":1.306}]},{"osd":2,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.72999999999999998}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.2609999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.89000000000000001}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":1.175}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.98599999999999999}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.22},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":1.345}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.84199999999999997}]},{"osd":1,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.749},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.98499999999999999}]},{"osd":2,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.63500000000000001}]},{"osd":3,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":1.0569999999999999}]},{"osd":4,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.878},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.097}]},{"osd":6,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.516,"5min":2.516,"15min":2.516},"last":1.137},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.94799999999999995}]},{"osd":7,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":1.401}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":153113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.497},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.753}]},{"osd":1,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.80800000000000005}]},{"osd":2,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":1.1200000000000001},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":1.0329999999999999}]},{"osd":3,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.85599999999999998}]},{"osd":5,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.64200000000000002}]},{"osd":6,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":1.167},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":1.097}]},{"osd":7,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":1.002},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":1.0629999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.39300000000000002}]},{"osd":1,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.38},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.29999999999999999}]},{"osd":2,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.36499999999999999}]},{"osd":4,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.79200000000000004}]},{"osd":5,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.73199999999999998}]},{"osd":6,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.63900000000000001}]},{"osd":7,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.86699999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":146998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.33300000000000002}]},{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.45300000000000001}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.51700000000000002}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.875}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.82599999999999996}]},{"osd":6,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.1989999999999998,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":0.78000000000000003}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.70599999999999996},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.90800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.57999999999999996}]},{"osd":2,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.98899999999999999}]},{"osd":3,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":1.1339999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.94299999999999995}]},{"osd":4,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.29}]},{"osd":5,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":1.222}]},{"osd":6,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.90100000000000002}]},{"osd":7,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":1.196},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.2609999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":974848,"data_stored":742358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.099}]},{"osd":2,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.91400000000000003}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.73299999999999998}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.0289999999999999}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.504}]},{"osd":6,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.169}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.1379999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-12T14:56:25.123 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-12T14:56:25.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.240+0000 7f8f7bad0640 1 -- 172.21.15.131:0/685911818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f740abae0 msgr2=0x7f8f740c9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:25.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.240+0000 7f8f7bad0640 1 --2- 172.21.15.131:0/685911818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f740abae0 0x7f8f740c9310 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8f74071110 tx=0x7f8f6801c810 comp rx=0 tx=0).stop 2023-09-12T14:56:25.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.240+0000 7f8f7bad0640 1 -- 172.21.15.131:0/685911818 shutdown_connections 2023-09-12T14:56:25.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.240+0000 7f8f7bad0640 1 --2- 172.21.15.131:0/685911818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f740cc430 0x7f8f740c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:25.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.240+0000 7f8f7bad0640 1 --2- 172.21.15.131:0/685911818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f740abae0 0x7f8f740c9310 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:25.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.240+0000 7f8f7bad0640 1 --2- 172.21.15.131:0/685911818 >> v2:172.21.15.187:3300/0 conn(0x7f8f74158d70 0x7f8f7406b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:25.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.240+0000 7f8f7bad0640 1 -- 172.21.15.131:0/685911818 >> 172.21.15.131:0/685911818 conn(0x7f8f74069840 msgr2=0x7f8f740cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:25.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.240+0000 7f8f7bad0640 1 -- 172.21.15.131:0/685911818 shutdown_connections 2023-09-12T14:56:25.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.240+0000 7f8f7bad0640 1 -- 172.21.15.131:0/685911818 wait complete. 2023-09-12T14:56:25.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.240+0000 7f8f7bad0640 1 Processor -- start 2023-09-12T14:56:25.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.240+0000 7f8f7bad0640 1 -- start start 2023-09-12T14:56:25.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.240+0000 7f8f7bad0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f740abae0 0x7f8f741f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:25.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.240+0000 7f8f7bad0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f740cc430 0x7f8f74139060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:25.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.240+0000 7f8f7bad0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f74158d70 0x7f8f74154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:25.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.240+0000 7f8f7bad0640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f74070d20 con 0x7f8f740abae0 2023-09-12T14:56:25.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.240+0000 7f8f7bad0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f741f8b00 con 0x7f8f740cc430 2023-09-12T14:56:25.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.240+0000 7f8f7bad0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f74153020 con 0x7f8f74158d70 2023-09-12T14:56:25.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.240+0000 7f8f7986c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f740abae0 0x7f8f741f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:25.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.240+0000 7f8f7986c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f740abae0 0x7f8f741f84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45068/0 (socket says 172.21.15.131:45068) 2023-09-12T14:56:25.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.240+0000 7f8f7986c640 1 -- 172.21.15.131:0/3610387679 learned_addr learned my addr 172.21.15.131:0/3610387679 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:25.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.244+0000 7f8f7a06d640 1 --2- 172.21.15.131:0/3610387679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f74158d70 0x7f8f74154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:25.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.244+0000 7f8f7906b640 1 --2- 172.21.15.131:0/3610387679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f740cc430 0x7f8f74139060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:25.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.244+0000 7f8f7986c640 1 -- 172.21.15.131:0/3610387679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f74158d70 msgr2=0x7f8f74154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:25.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.244+0000 7f8f7986c640 1 --2- 172.21.15.131:0/3610387679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f74158d70 0x7f8f74154a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:25.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.244+0000 7f8f7986c640 1 -- 172.21.15.131:0/3610387679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f740cc430 msgr2=0x7f8f74139060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:25.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.244+0000 7f8f7986c640 1 --2- 172.21.15.131:0/3610387679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f740cc430 0x7f8f74139060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:25.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.244+0000 7f8f7986c640 1 -- 172.21.15.131:0/3610387679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f68009560 con 0x7f8f740abae0 2023-09-12T14:56:25.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.244+0000 7f8f7a06d640 1 --2- 172.21.15.131:0/3610387679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f74158d70 0x7f8f74154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:25.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.244+0000 7f8f7986c640 1 --2- 172.21.15.131:0/3610387679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f740abae0 0x7f8f741f84d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8f640058d0 tx=0x7f8f6400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:25.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.244+0000 7f8f727fc640 1 -- 172.21.15.131:0/3610387679 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f64009c30 con 0x7f8f740abae0 2023-09-12T14:56:25.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.244+0000 7f8f727fc640 1 -- 172.21.15.131:0/3610387679 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8f64009e80 con 0x7f8f740abae0 2023-09-12T14:56:25.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.244+0000 7f8f727fc640 1 -- 172.21.15.131:0/3610387679 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f640106a0 con 0x7f8f740abae0 2023-09-12T14:56:25.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.244+0000 7f8f7bad0640 1 -- 172.21.15.131:0/3610387679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f741395a0 con 0x7f8f740abae0 2023-09-12T14:56:25.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.244+0000 7f8f7bad0640 1 -- 172.21.15.131:0/3610387679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f74152bc0 con 0x7f8f740abae0 2023-09-12T14:56:25.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.244+0000 7f8f7bad0640 1 -- 172.21.15.131:0/3610387679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f74152df0 con 0x7f8f740abae0 2023-09-12T14:56:25.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.244+0000 7f8f727fc640 1 -- 172.21.15.131:0/3610387679 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f8f64010800 con 0x7f8f740abae0 2023-09-12T14:56:25.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.244+0000 7f8f727fc640 1 --2- 172.21.15.131:0/3610387679 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8f60065b20 0x7f8f60067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:25.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.244+0000 7f8f727fc640 1 -- 172.21.15.131:0/3610387679 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8f64083510 con 0x7f8f740abae0 2023-09-12T14:56:25.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.244+0000 7f8f7906b640 1 --2- 172.21.15.131:0/3610387679 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8f60065b20 0x7f8f60067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:25.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.244+0000 7f8f7906b640 1 --2- 172.21.15.131:0/3610387679 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8f60065b20 0x7f8f60067fe0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8f680ae000 tx=0x7f8f6801b050 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:25.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.248+0000 7f8f727fc640 1 -- 172.21.15.131:0/3610387679 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f8f74152df0 con 0x7f8f740abae0 2023-09-12T14:56:25.405 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.404+0000 7f8f7bad0640 1 -- 172.21.15.131:0/3610387679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f74152df0 con 0x7f8f740abae0 2023-09-12T14:56:25.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.404+0000 7f8f727fc640 1 -- 172.21.15.131:0/3610387679 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+13022 (secure 0 0 0) 0x7f8f74152df0 con 0x7f8f740abae0 2023-09-12T14:56:25.407 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:56:25.407 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":36,"fsid":"6efb91e0-47b4-40bb-ae12-7c7ea0e893df","created":"2023-09-12T14:54:32.153202+0000","modified":"2023-09-12T14:56:22.280046+0000","last_up_change":"2023-09-12T14:54:44.786814+0000","last_in_change":"2023-09-12T14:54:35.359818+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-12T14:54:45.654004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ddb30e8-fd9b-46c1-a667-0eaba00cdc18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6801","nonce":2063242002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6803","nonce":2063242002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6807","nonce":2063242002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2063242002},{"type":"v1","addr":"172.21.15.131:6805","nonce":2063242002}]},"public_addr":"172.21.15.131:6801/2063242002","cluster_addr":"172.21.15.131:6803/2063242002","heartbeat_back_addr":"172.21.15.131:6807/2063242002","heartbeat_front_addr":"172.21.15.131:6805/2063242002","state":["exists","up"]},{"osd":1,"uuid":"095cca20-beb2-4d55-b8f8-f1910692ac5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6809","nonce":3882516365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6811","nonce":3882516365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6815","nonce":3882516365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3882516365},{"type":"v1","addr":"172.21.15.131:6813","nonce":3882516365}]},"public_addr":"172.21.15.131:6809/3882516365","cluster_addr":"172.21.15.131:6811/3882516365","heartbeat_back_addr":"172.21.15.131:6815/3882516365","heartbeat_front_addr":"172.21.15.131:6813/3882516365","state":["exists","up"]},{"osd":2,"uuid":"14ba94e7-71fd-48c3-a92e-04b6274992fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6825","nonce":3891085905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6827","nonce":3891085905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6831","nonce":3891085905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":3891085905},{"type":"v1","addr":"172.21.15.131:6829","nonce":3891085905}]},"public_addr":"172.21.15.131:6825/3891085905","cluster_addr":"172.21.15.131:6827/3891085905","heartbeat_back_addr":"172.21.15.131:6831/3891085905","heartbeat_front_addr":"172.21.15.131:6829/3891085905","state":["exists","up"]},{"osd":3,"uuid":"85179676-991f-4155-9378-73d9e6fed59e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6817","nonce":2354456273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6819","nonce":2354456273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6823","nonce":2354456273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":2354456273},{"type":"v1","addr":"172.21.15.131:6821","nonce":2354456273}]},"public_addr":"172.21.15.131:6817/2354456273","cluster_addr":"172.21.15.131:6819/2354456273","heartbeat_back_addr":"172.21.15.131:6823/2354456273","heartbeat_front_addr":"172.21.15.131:6821/2354456273","state":["exists","up"]},{"osd":4,"uuid":"1b65089f-9b5f-4133-8997-e06bf12287a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6802","nonce":1649621344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6806","nonce":1649621344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6814","nonce":1649621344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1649621344},{"type":"v1","addr":"172.21.15.187:6810","nonce":1649621344}]},"public_addr":"172.21.15.187:6802/1649621344","cluster_addr":"172.21.15.187:6806/1649621344","heartbeat_back_addr":"172.21.15.187:6814/1649621344","heartbeat_front_addr":"172.21.15.187:6810/1649621344","state":["exists","up"]},{"osd":5,"uuid":"cd4abd3e-97a3-4bd6-82b9-c531b6cf266c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6825","nonce":2170805852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6827","nonce":2170805852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6831","nonce":2170805852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2170805852},{"type":"v1","addr":"172.21.15.187:6829","nonce":2170805852}]},"public_addr":"172.21.15.187:6825/2170805852","cluster_addr":"172.21.15.187:6827/2170805852","heartbeat_back_addr":"172.21.15.187:6831/2170805852","heartbeat_front_addr":"172.21.15.187:6829/2170805852","state":["exists","up"]},{"osd":6,"uuid":"53eed235-0d36-4830-a8c0-c2e19fe46d46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6803","nonce":221066879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6807","nonce":221066879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6815","nonce":221066879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":221066879},{"type":"v1","addr":"172.21.15.187:6811","nonce":221066879}]},"public_addr":"172.21.15.187:6803/221066879","cluster_addr":"172.21.15.187:6807/221066879","heartbeat_back_addr":"172.21.15.187:6815/221066879","heartbeat_front_addr":"172.21.15.187:6811/221066879","state":["exists","up"]},{"osd":7,"uuid":"9d511448-aac0-4c4b-99ed-221457021871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6817","nonce":827490193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6819","nonce":827490193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6823","nonce":827490193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":827490193},{"type":"v1","addr":"172.21.15.187:6821","nonce":827490193}]},"public_addr":"172.21.15.187:6817/827490193","cluster_addr":"172.21.15.187:6819/827490193","heartbeat_back_addr":"172.21.15.187:6823/827490193","heartbeat_front_addr":"172.21.15.187:6821/827490193","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.682201+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.671911+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.841576+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.683647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.011637+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.988152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:40.865107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-12T14:54:41.005285+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6835/871788191":"2023-09-13T14:56:19.261785+0000","172.21.15.131:6834/871788191":"2023-09-13T14:56:19.261785+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-12T14:56:25.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.404+0000 7f8f7bad0640 1 -- 172.21.15.131:0/3610387679 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8f60065b20 msgr2=0x7f8f60067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:25.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.404+0000 7f8f7bad0640 1 --2- 172.21.15.131:0/3610387679 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8f60065b20 0x7f8f60067fe0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8f680ae000 tx=0x7f8f6801b050 comp rx=0 tx=0).stop 2023-09-12T14:56:25.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.404+0000 7f8f7bad0640 1 -- 172.21.15.131:0/3610387679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f740abae0 msgr2=0x7f8f741f84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:25.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.404+0000 7f8f7bad0640 1 --2- 172.21.15.131:0/3610387679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f740abae0 0x7f8f741f84d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8f640058d0 tx=0x7f8f6400dde0 comp rx=0 tx=0).stop 2023-09-12T14:56:25.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.404+0000 7f8f7bad0640 1 -- 172.21.15.131:0/3610387679 shutdown_connections 2023-09-12T14:56:25.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.404+0000 7f8f7bad0640 1 --2- 172.21.15.131:0/3610387679 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f8f60065b20 0x7f8f60067fe0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:25.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.404+0000 7f8f7bad0640 1 --2- 172.21.15.131:0/3610387679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f74158d70 0x7f8f74154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:25.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.404+0000 7f8f7bad0640 1 --2- 172.21.15.131:0/3610387679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f740cc430 0x7f8f74139060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:25.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.404+0000 7f8f7bad0640 1 --2- 172.21.15.131:0/3610387679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f740abae0 0x7f8f741f84d0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:25.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.404+0000 7f8f7bad0640 1 -- 172.21.15.131:0/3610387679 >> 172.21.15.131:0/3610387679 conn(0x7f8f74069840 msgr2=0x7f8f740ac7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:25.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.408+0000 7f8f7bad0640 1 -- 172.21.15.131:0/3610387679 shutdown_connections 2023-09-12T14:56:25.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:25.408+0000 7f8f7bad0640 1 -- 172.21.15.131:0/3610387679 wait complete. 2023-09-12T14:56:25.726 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-12T14:56:25.729 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-12T14:56:26.425 INFO:tasks.ceph:Scrubbing osd.0 2023-09-12T14:56:26.426 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-09-12T14:56:26.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.564+0000 7f5780342640 1 -- 172.21.15.131:0/1032559703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5778158180 msgr2=0x7f57780cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:26.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.564+0000 7f5780342640 1 --2- 172.21.15.131:0/1032559703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5778158180 0x7f57780cae70 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f57780cd230 tx=0x7f576c01c830 comp rx=0 tx=0).stop 2023-09-12T14:56:26.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.564+0000 7f5780342640 1 -- 172.21.15.131:0/1032559703 shutdown_connections 2023-09-12T14:56:26.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.564+0000 7f5780342640 1 --2- 172.21.15.131:0/1032559703 >> v2:172.21.15.187:3300/0 conn(0x7f577806b050 0x7f577806b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:26.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.564+0000 7f5780342640 1 --2- 172.21.15.131:0/1032559703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5778158180 0x7f57780cae70 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:26.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.564+0000 7f5780342640 1 -- 172.21.15.131:0/1032559703 >> 172.21.15.131:0/1032559703 conn(0x7f5778069840 msgr2=0x7f57780cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:26.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.564+0000 7f5780342640 1 -- 172.21.15.131:0/1032559703 shutdown_connections 2023-09-12T14:56:26.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.564+0000 7f5780342640 1 -- 172.21.15.131:0/1032559703 wait complete. 2023-09-12T14:56:26.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.564+0000 7f5780342640 1 Processor -- start 2023-09-12T14:56:26.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.564+0000 7f5780342640 1 -- start start 2023-09-12T14:56:26.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f5780342640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f577806b050 0x7f57781460d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:26.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f5780342640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57780ab730 0x7f57781f1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:26.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f5780342640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5778158180 0x7f577814d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:26.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f5780342640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5778146cd0 con 0x7f57780ab730 2023-09-12T14:56:26.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f5780342640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5778148190 con 0x7f5778158180 2023-09-12T14:56:26.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f5780342640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5778148440 con 0x7f577806b050 2023-09-12T14:56:26.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f577d8dd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57780ab730 0x7f57781f1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:26.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f577d8dd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57780ab730 0x7f57781f1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45098/0 (socket says 172.21.15.131:45098) 2023-09-12T14:56:26.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f577d8dd640 1 -- 172.21.15.131:0/1105674050 learned_addr learned my addr 172.21.15.131:0/1105674050 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:26.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f577d8dd640 1 -- 172.21.15.131:0/1105674050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f577806b050 msgr2=0x7f57781460d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:26.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f577e8df640 1 --2- 172.21.15.131:0/1105674050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5778158180 0x7f577814d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:26.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f577d8dd640 1 --2- 172.21.15.131:0/1105674050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f577806b050 0x7f57781460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:26.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f577d8dd640 1 -- 172.21.15.131:0/1105674050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5778158180 msgr2=0x7f577814d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:26.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f577d8dd640 1 --2- 172.21.15.131:0/1105674050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5778158180 0x7f577814d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:26.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f577d8dd640 1 -- 172.21.15.131:0/1105674050 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576c009560 con 0x7f57780ab730 2023-09-12T14:56:26.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f577d8dd640 1 --2- 172.21.15.131:0/1105674050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57780ab730 0x7f57781f1f60 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f576c000c10 tx=0x7f576c0a5790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:26.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f5776ffd640 1 -- 172.21.15.131:0/1105674050 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c005230 con 0x7f57780ab730 2023-09-12T14:56:26.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f5776ffd640 1 -- 172.21.15.131:0/1105674050 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f576c0053b0 con 0x7f57780ab730 2023-09-12T14:56:26.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f5780342640 1 -- 172.21.15.131:0/1105674050 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5778148720 con 0x7f57780ab730 2023-09-12T14:56:26.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f5776ffd640 1 -- 172.21.15.131:0/1105674050 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c004930 con 0x7f57780ab730 2023-09-12T14:56:26.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f5780342640 1 -- 172.21.15.131:0/1105674050 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f577814b520 con 0x7f57780ab730 2023-09-12T14:56:26.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f5780342640 1 -- 172.21.15.131:0/1105674050 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5778073580 con 0x7f57780ab730 2023-09-12T14:56:26.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f5776ffd640 1 -- 172.21.15.131:0/1105674050 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f576c0b4070 con 0x7f57780ab730 2023-09-12T14:56:26.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f5776ffd640 1 --2- 172.21.15.131:0/1105674050 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5764065b20 0x7f5764067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:26.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f5776ffd640 1 -- 172.21.15.131:0/1105674050 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f576c11d930 con 0x7f57780ab730 2023-09-12T14:56:26.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f577e0de640 1 --2- 172.21.15.131:0/1105674050 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5764065b20 0x7f5764067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:26.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f5776ffd640 1 --2- 172.21.15.131:0/1105674050 >> [v2:172.21.15.131:6800/2063242002,v1:172.21.15.131:6801/2063242002] conn(0x7f576406b980 0x7f576406dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:26.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f5776ffd640 1 -- 172.21.15.131:0/1105674050 --> [v2:172.21.15.131:6800/2063242002,v1:172.21.15.131:6801/2063242002] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f576406e450 con 0x7f576406b980 2023-09-12T14:56:26.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f5776ffd640 1 -- 172.21.15.131:0/1105674050 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f576c11dd80 con 0x7f57780ab730 2023-09-12T14:56:26.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f577e8df640 1 --2- 172.21.15.131:0/1105674050 >> [v2:172.21.15.131:6800/2063242002,v1:172.21.15.131:6801/2063242002] conn(0x7f576406b980 0x7f576406dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:26.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f577e0de640 1 --2- 172.21.15.131:0/1105674050 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5764065b20 0x7f5764067fe0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5768007fd0 tx=0x7f5768007d40 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:26.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.568+0000 7f577e8df640 1 --2- 172.21.15.131:0/1105674050 >> [v2:172.21.15.131:6800/2063242002,v1:172.21.15.131:6801/2063242002] conn(0x7f576406b980 0x7f576406dda0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:26.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.572+0000 7f5776ffd640 1 -- 172.21.15.131:0/1105674050 <== osd.0 v2:172.21.15.131:6800/2063242002 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f576406e450 con 0x7f576406b980 2023-09-12T14:56:26.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.584+0000 7f5780342640 1 -- 172.21.15.131:0/1105674050 --> [v2:172.21.15.131:6800/2063242002,v1:172.21.15.131:6801/2063242002] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f577814f790 con 0x7f576406b980 2023-09-12T14:56:26.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.584+0000 7f5776ffd640 1 -- 172.21.15.131:0/1105674050 <== osd.0 v2:172.21.15.131:6800/2063242002 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f577814f790 con 0x7f576406b980 2023-09-12T14:56:26.587 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-09-12T14:56:26.587 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-12T14:56:26.587 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-09-12T14:56:26.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.584+0000 7f5780342640 1 -- 172.21.15.131:0/1105674050 >> [v2:172.21.15.131:6800/2063242002,v1:172.21.15.131:6801/2063242002] conn(0x7f576406b980 msgr2=0x7f576406dda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:26.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.584+0000 7f5780342640 1 --2- 172.21.15.131:0/1105674050 >> [v2:172.21.15.131:6800/2063242002,v1:172.21.15.131:6801/2063242002] conn(0x7f576406b980 0x7f576406dda0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:26.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.584+0000 7f5780342640 1 -- 172.21.15.131:0/1105674050 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5764065b20 msgr2=0x7f5764067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:26.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.584+0000 7f5780342640 1 --2- 172.21.15.131:0/1105674050 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5764065b20 0x7f5764067fe0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5768007fd0 tx=0x7f5768007d40 comp rx=0 tx=0).stop 2023-09-12T14:56:26.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.584+0000 7f5780342640 1 -- 172.21.15.131:0/1105674050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57780ab730 msgr2=0x7f57781f1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:26.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.584+0000 7f5780342640 1 --2- 172.21.15.131:0/1105674050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57780ab730 0x7f57781f1f60 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f576c000c10 tx=0x7f576c0a5790 comp rx=0 tx=0).stop 2023-09-12T14:56:26.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.584+0000 7f577e8df640 1 -- 172.21.15.131:0/1105674050 reap_dead start 2023-09-12T14:56:26.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.584+0000 7f5780342640 1 -- 172.21.15.131:0/1105674050 shutdown_connections 2023-09-12T14:56:26.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.584+0000 7f5780342640 1 -- 172.21.15.131:0/1105674050 >> 172.21.15.131:0/1105674050 conn(0x7f5778069840 msgr2=0x7f577814e970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:26.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.584+0000 7f5780342640 1 -- 172.21.15.131:0/1105674050 shutdown_connections 2023-09-12T14:56:26.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.584+0000 7f5780342640 1 -- 172.21.15.131:0/1105674050 wait complete. 2023-09-12T14:56:26.604 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-09-12T14:56:26.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.720+0000 7f1d8f7fc640 1 -- 172.21.15.131:0/1414784296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d880cf2d0 msgr2=0x7f1d8815e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:26.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.720+0000 7f1d8f7fc640 1 --2- 172.21.15.131:0/1414784296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d880cf2d0 0x7f1d8815e2e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1d880b06e0 tx=0x7f1d8401c7e0 comp rx=0 tx=0).stop 2023-09-12T14:56:26.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.720+0000 7f1d8f7fc640 1 -- 172.21.15.131:0/1414784296 shutdown_connections 2023-09-12T14:56:26.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.720+0000 7f1d8f7fc640 1 --2- 172.21.15.131:0/1414784296 >> v2:172.21.15.187:3300/0 conn(0x7f1d880b1760 0x7f1d8815e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:26.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.720+0000 7f1d8f7fc640 1 --2- 172.21.15.131:0/1414784296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d880cf2d0 0x7f1d8815e2e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:26.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.720+0000 7f1d8f7fc640 1 -- 172.21.15.131:0/1414784296 >> 172.21.15.131:0/1414784296 conn(0x7f1d8809a200 msgr2=0x7f1d880d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:26.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.720+0000 7f1d8f7fc640 1 -- 172.21.15.131:0/1414784296 shutdown_connections 2023-09-12T14:56:26.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.720+0000 7f1d8f7fc640 1 -- 172.21.15.131:0/1414784296 wait complete. 2023-09-12T14:56:26.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.720+0000 7f1d8f7fc640 1 Processor -- start 2023-09-12T14:56:26.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.720+0000 7f1d8f7fc640 1 -- start start 2023-09-12T14:56:26.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.720+0000 7f1d8f7fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d880b1760 0x7f1d880c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:26.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d8f7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d880cf2d0 0x7f1d880b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:26.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d8f7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d880d1680 0x7f1d881596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:26.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d8f7fc640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d880c8250 con 0x7f1d880d1680 2023-09-12T14:56:26.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d8f7fc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d880b43b0 con 0x7f1d880cf2d0 2023-09-12T14:56:26.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d8f7fc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d880b4690 con 0x7f1d880b1760 2023-09-12T14:56:26.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d8dd99640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d880d1680 0x7f1d881596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:26.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d8dd99640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d880d1680 0x7f1d881596b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45126/0 (socket says 172.21.15.131:45126) 2023-09-12T14:56:26.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d8dd99640 1 -- 172.21.15.131:0/1129431877 learned_addr learned my addr 172.21.15.131:0/1129431877 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:26.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d8cd97640 1 --2- 172.21.15.131:0/1129431877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d880cf2d0 0x7f1d880b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:26.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d8d598640 1 --2- 172.21.15.131:0/1129431877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d880b1760 0x7f1d880c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:26.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d8dd99640 1 -- 172.21.15.131:0/1129431877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d880b1760 msgr2=0x7f1d880c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:26.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d8dd99640 1 --2- 172.21.15.131:0/1129431877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d880b1760 0x7f1d880c7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:26.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d8dd99640 1 -- 172.21.15.131:0/1129431877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d880cf2d0 msgr2=0x7f1d880b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:26.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d8dd99640 1 --2- 172.21.15.131:0/1129431877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d880cf2d0 0x7f1d880b3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:26.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d8dd99640 1 -- 172.21.15.131:0/1129431877 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d840094a0 con 0x7f1d880d1680 2023-09-12T14:56:26.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d8cd97640 1 --2- 172.21.15.131:0/1129431877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d880cf2d0 0x7f1d880b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:26.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d8dd99640 1 --2- 172.21.15.131:0/1129431877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d880d1680 0x7f1d881596b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000ea20 tx=0x7f1d8000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:26.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d7dffb640 1 -- 172.21.15.131:0/1129431877 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8000cde0 con 0x7f1d880d1680 2023-09-12T14:56:26.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d7dffb640 1 -- 172.21.15.131:0/1129431877 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1d80004c40 con 0x7f1d880d1680 2023-09-12T14:56:26.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d8f7fc640 1 -- 172.21.15.131:0/1129431877 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d880b4970 con 0x7f1d880d1680 2023-09-12T14:56:26.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d7dffb640 1 -- 172.21.15.131:0/1129431877 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d80005020 con 0x7f1d880d1680 2023-09-12T14:56:26.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d8f7fc640 1 -- 172.21.15.131:0/1129431877 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d880b40a0 con 0x7f1d880d1680 2023-09-12T14:56:26.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d8f7fc640 1 -- 172.21.15.131:0/1129431877 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d88073580 con 0x7f1d880d1680 2023-09-12T14:56:26.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d7dffb640 1 -- 172.21.15.131:0/1129431877 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f1d80005180 con 0x7f1d880d1680 2023-09-12T14:56:26.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d7dffb640 1 --2- 172.21.15.131:0/1129431877 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1d6c065b20 0x7f1d6c067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:26.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d7dffb640 1 -- 172.21.15.131:0/1129431877 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1d80083aa0 con 0x7f1d880d1680 2023-09-12T14:56:26.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d8d598640 1 --2- 172.21.15.131:0/1129431877 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1d6c065b20 0x7f1d6c067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:26.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.724+0000 7f1d8d598640 1 --2- 172.21.15.131:0/1129431877 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1d6c065b20 0x7f1d6c067fe0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1d840af000 tx=0x7f1d8401b020 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:26.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.728+0000 7f1d7dffb640 1 -- 172.21.15.131:0/1129431877 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f1d800516f0 con 0x7f1d880d1680 2023-09-12T14:56:26.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.880+0000 7f1d8f7fc640 1 -- 172.21.15.131:0/1129431877 --> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d88167f40 con 0x7f1d6c065b20 2023-09-12T14:56:26.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.884+0000 7f1d7dffb640 1 -- 172.21.15.131:0/1129431877 <== mgr.4200 v2:172.21.15.131:6832/2298103651 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1d84005ca0 con 0x7f1d6c065b20 2023-09-12T14:56:26.886 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 0 to deep-scrub 2023-09-12T14:56:26.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.888+0000 7f1d8f7fc640 1 -- 172.21.15.131:0/1129431877 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1d6c065b20 msgr2=0x7f1d6c067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:26.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.888+0000 7f1d8f7fc640 1 --2- 172.21.15.131:0/1129431877 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1d6c065b20 0x7f1d6c067fe0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1d840af000 tx=0x7f1d8401b020 comp rx=0 tx=0).stop 2023-09-12T14:56:26.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.888+0000 7f1d8f7fc640 1 -- 172.21.15.131:0/1129431877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d880d1680 msgr2=0x7f1d881596b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:26.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.888+0000 7f1d8f7fc640 1 --2- 172.21.15.131:0/1129431877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d880d1680 0x7f1d881596b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000ea20 tx=0x7f1d8000eee0 comp rx=0 tx=0).stop 2023-09-12T14:56:26.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.888+0000 7f1d8f7fc640 1 -- 172.21.15.131:0/1129431877 shutdown_connections 2023-09-12T14:56:26.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.888+0000 7f1d8f7fc640 1 --2- 172.21.15.131:0/1129431877 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1d6c065b20 0x7f1d6c067fe0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:26.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.888+0000 7f1d8f7fc640 1 --2- 172.21.15.131:0/1129431877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d880d1680 0x7f1d881596b0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:26.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.888+0000 7f1d8f7fc640 1 --2- 172.21.15.131:0/1129431877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d880cf2d0 0x7f1d880b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:26.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.888+0000 7f1d8f7fc640 1 --2- 172.21.15.131:0/1129431877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d880b1760 0x7f1d880c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:26.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.888+0000 7f1d8f7fc640 1 -- 172.21.15.131:0/1129431877 >> 172.21.15.131:0/1129431877 conn(0x7f1d8809a200 msgr2=0x7f1d880cc350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:26.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.888+0000 7f1d8f7fc640 1 -- 172.21.15.131:0/1129431877 shutdown_connections 2023-09-12T14:56:26.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:26.888+0000 7f1d8f7fc640 1 -- 172.21.15.131:0/1129431877 wait complete. 2023-09-12T14:56:26.909 INFO:tasks.ceph:Scrubbing osd.1 2023-09-12T14:56:26.910 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-09-12T14:56:27.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.040+0000 7ff6dc449640 1 -- 172.21.15.131:0/578885687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6d41553b0 msgr2=0x7ff6d40afa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.040+0000 7ff6dc449640 1 --2- 172.21.15.131:0/578885687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6d41553b0 0x7ff6d40afa30 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc009c80 tx=0x7ff6cc01cd80 comp rx=0 tx=0).stop 2023-09-12T14:56:27.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.040+0000 7ff6dc449640 1 -- 172.21.15.131:0/578885687 shutdown_connections 2023-09-12T14:56:27.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.040+0000 7ff6dc449640 1 --2- 172.21.15.131:0/578885687 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6d40aff70 0x7ff6d40cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.040+0000 7ff6dc449640 1 --2- 172.21.15.131:0/578885687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6d41553b0 0x7ff6d40afa30 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.040+0000 7ff6dc449640 1 -- 172.21.15.131:0/578885687 >> 172.21.15.131:0/578885687 conn(0x7ff6d406a520 msgr2=0x7ff6d406a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:27.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.040+0000 7ff6dc449640 1 -- 172.21.15.131:0/578885687 shutdown_connections 2023-09-12T14:56:27.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.040+0000 7ff6dc449640 1 -- 172.21.15.131:0/578885687 wait complete. 2023-09-12T14:56:27.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.040+0000 7ff6dc449640 1 Processor -- start 2023-09-12T14:56:27.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.040+0000 7ff6dc449640 1 -- start start 2023-09-12T14:56:27.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.040+0000 7ff6dc449640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6d40aff70 0x7ff6d41f58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:27.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.040+0000 7ff6dc449640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6d40cc450 0x7ff6d413cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:27.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.040+0000 7ff6dc449640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6d41553b0 0x7ff6d4154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:27.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.040+0000 7ff6dc449640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d40c3460 con 0x7ff6d40cc450 2023-09-12T14:56:27.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.040+0000 7ff6dc449640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d4154c70 con 0x7ff6d40aff70 2023-09-12T14:56:27.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.040+0000 7ff6dc449640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d41f67e0 con 0x7ff6d41553b0 2023-09-12T14:56:27.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.040+0000 7ff6d99e4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6d40cc450 0x7ff6d413cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:27.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.040+0000 7ff6d99e4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6d40cc450 0x7ff6d413cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45144/0 (socket says 172.21.15.131:45144) 2023-09-12T14:56:27.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.040+0000 7ff6d99e4640 1 -- 172.21.15.131:0/3759150602 learned_addr learned my addr 172.21.15.131:0/3759150602 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:27.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.044+0000 7ff6da9e6640 1 --2- 172.21.15.131:0/3759150602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6d41553b0 0x7ff6d4154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:27.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.044+0000 7ff6da1e5640 1 --2- 172.21.15.131:0/3759150602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6d40aff70 0x7ff6d41f58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:27.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.044+0000 7ff6d99e4640 1 -- 172.21.15.131:0/3759150602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6d41553b0 msgr2=0x7ff6d4154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.044+0000 7ff6d99e4640 1 --2- 172.21.15.131:0/3759150602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6d41553b0 0x7ff6d4154730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.044+0000 7ff6d99e4640 1 -- 172.21.15.131:0/3759150602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6d40aff70 msgr2=0x7ff6d41f58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.044+0000 7ff6d99e4640 1 --2- 172.21.15.131:0/3759150602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6d40aff70 0x7ff6d41f58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.044+0000 7ff6d99e4640 1 -- 172.21.15.131:0/3759150602 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc0098e0 con 0x7ff6d40cc450 2023-09-12T14:56:27.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.044+0000 7ff6da1e5640 1 --2- 172.21.15.131:0/3759150602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6d40aff70 0x7ff6d41f58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:27.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.044+0000 7ff6da9e6640 1 --2- 172.21.15.131:0/3759150602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6d41553b0 0x7ff6d4154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:27.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.044+0000 7ff6d99e4640 1 --2- 172.21.15.131:0/3759150602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6d40cc450 0x7ff6d413cf10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400b550 tx=0x7ff6c400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:27.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.044+0000 7ff6caffd640 1 -- 172.21.15.131:0/3759150602 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4010870 con 0x7ff6d40cc450 2023-09-12T14:56:27.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.044+0000 7ff6caffd640 1 -- 172.21.15.131:0/3759150602 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff6c4010ae0 con 0x7ff6d40cc450 2023-09-12T14:56:27.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.044+0000 7ff6dc449640 1 -- 172.21.15.131:0/3759150602 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d41f5f40 con 0x7ff6d40cc450 2023-09-12T14:56:27.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.044+0000 7ff6caffd640 1 -- 172.21.15.131:0/3759150602 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c400f3c0 con 0x7ff6d40cc450 2023-09-12T14:56:27.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.044+0000 7ff6dc449640 1 -- 172.21.15.131:0/3759150602 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d41f6500 con 0x7ff6d40cc450 2023-09-12T14:56:27.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.044+0000 7ff6dc449640 1 -- 172.21.15.131:0/3759150602 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff6d4155950 con 0x7ff6d40cc450 2023-09-12T14:56:27.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.044+0000 7ff6caffd640 1 -- 172.21.15.131:0/3759150602 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7ff6c400f520 con 0x7ff6d40cc450 2023-09-12T14:56:27.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.044+0000 7ff6caffd640 1 --2- 172.21.15.131:0/3759150602 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7ff6b8065b20 0x7ff6b8067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:27.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.044+0000 7ff6caffd640 1 -- 172.21.15.131:0/3759150602 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff6c4083250 con 0x7ff6d40cc450 2023-09-12T14:56:27.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.044+0000 7ff6da1e5640 1 --2- 172.21.15.131:0/3759150602 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7ff6b8065b20 0x7ff6b8067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:27.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.044+0000 7ff6caffd640 1 --2- 172.21.15.131:0/3759150602 >> [v2:172.21.15.131:6808/3882516365,v1:172.21.15.131:6809/3882516365] conn(0x7ff6b806b980 0x7ff6b806dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:27.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.044+0000 7ff6caffd640 1 -- 172.21.15.131:0/3759150602 --> [v2:172.21.15.131:6808/3882516365,v1:172.21.15.131:6809/3882516365] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff6b806e450 con 0x7ff6b806b980 2023-09-12T14:56:27.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.044+0000 7ff6caffd640 1 -- 172.21.15.131:0/3759150602 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6c4085910 con 0x7ff6d40cc450 2023-09-12T14:56:27.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.044+0000 7ff6da9e6640 1 --2- 172.21.15.131:0/3759150602 >> [v2:172.21.15.131:6808/3882516365,v1:172.21.15.131:6809/3882516365] conn(0x7ff6b806b980 0x7ff6b806dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:27.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.044+0000 7ff6da1e5640 1 --2- 172.21.15.131:0/3759150602 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7ff6b8065b20 0x7ff6b8067fe0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0af000 tx=0x7ff6cc01b5c0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:27.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.044+0000 7ff6da9e6640 1 --2- 172.21.15.131:0/3759150602 >> [v2:172.21.15.131:6808/3882516365,v1:172.21.15.131:6809/3882516365] conn(0x7ff6b806b980 0x7ff6b806dda0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:27.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.044+0000 7ff6caffd640 1 -- 172.21.15.131:0/3759150602 <== osd.1 v2:172.21.15.131:6808/3882516365 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7ff6b806e450 con 0x7ff6b806b980 2023-09-12T14:56:27.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.060+0000 7ff6dc449640 1 -- 172.21.15.131:0/3759150602 --> [v2:172.21.15.131:6808/3882516365,v1:172.21.15.131:6809/3882516365] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff6d4073580 con 0x7ff6b806b980 2023-09-12T14:56:27.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.060+0000 7ff6caffd640 1 -- 172.21.15.131:0/3759150602 <== osd.1 v2:172.21.15.131:6808/3882516365 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff6d4073580 con 0x7ff6b806b980 2023-09-12T14:56:27.063 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-09-12T14:56:27.063 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-12T14:56:27.063 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-09-12T14:56:27.064 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.060+0000 7ff6dc449640 1 -- 172.21.15.131:0/3759150602 >> [v2:172.21.15.131:6808/3882516365,v1:172.21.15.131:6809/3882516365] conn(0x7ff6b806b980 msgr2=0x7ff6b806dda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.060+0000 7ff6dc449640 1 --2- 172.21.15.131:0/3759150602 >> [v2:172.21.15.131:6808/3882516365,v1:172.21.15.131:6809/3882516365] conn(0x7ff6b806b980 0x7ff6b806dda0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.060+0000 7ff6dc449640 1 -- 172.21.15.131:0/3759150602 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7ff6b8065b20 msgr2=0x7ff6b8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.060+0000 7ff6dc449640 1 --2- 172.21.15.131:0/3759150602 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7ff6b8065b20 0x7ff6b8067fe0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0af000 tx=0x7ff6cc01b5c0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.060+0000 7ff6dc449640 1 -- 172.21.15.131:0/3759150602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6d40cc450 msgr2=0x7ff6d413cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.060+0000 7ff6dc449640 1 --2- 172.21.15.131:0/3759150602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6d40cc450 0x7ff6d413cf10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400b550 tx=0x7ff6c400ba10 comp rx=0 tx=0).stop 2023-09-12T14:56:27.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.060+0000 7ff6da9e6640 1 -- 172.21.15.131:0/3759150602 reap_dead start 2023-09-12T14:56:27.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.060+0000 7ff6dc449640 1 -- 172.21.15.131:0/3759150602 shutdown_connections 2023-09-12T14:56:27.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.060+0000 7ff6dc449640 1 -- 172.21.15.131:0/3759150602 >> 172.21.15.131:0/3759150602 conn(0x7ff6d406a520 msgr2=0x7ff6d4152120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:27.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.060+0000 7ff6dc449640 1 -- 172.21.15.131:0/3759150602 shutdown_connections 2023-09-12T14:56:27.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.060+0000 7ff6dc449640 1 -- 172.21.15.131:0/3759150602 wait complete. 2023-09-12T14:56:27.078 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-09-12T14:56:27.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.196+0000 7f20bd6d5640 1 -- 172.21.15.131:0/995970988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20b81445d0 msgr2=0x7f20b806be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.196+0000 7f20bd6d5640 1 --2- 172.21.15.131:0/995970988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20b81445d0 0x7f20b806be10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009bc0 tx=0x7f20b001cac0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.196+0000 7f20bd6d5640 1 -- 172.21.15.131:0/995970988 shutdown_connections 2023-09-12T14:56:27.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.196+0000 7f20bd6d5640 1 --2- 172.21.15.131:0/995970988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20b81445d0 0x7f20b806be10 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.196+0000 7f20bd6d5640 1 -- 172.21.15.131:0/995970988 >> 172.21.15.131:0/995970988 conn(0x7f20b806a520 msgr2=0x7f20b806a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:27.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.196+0000 7f20bd6d5640 1 -- 172.21.15.131:0/995970988 shutdown_connections 2023-09-12T14:56:27.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.196+0000 7f20bd6d5640 1 -- 172.21.15.131:0/995970988 wait complete. 2023-09-12T14:56:27.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.196+0000 7f20bd6d5640 1 Processor -- start 2023-09-12T14:56:27.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.196+0000 7f20bd6d5640 1 -- start start 2023-09-12T14:56:27.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.196+0000 7f20bd6d5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20b8070840 0x7f20b8159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:27.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.196+0000 7f20bd6d5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20b80cd3a0 0x7f20b80b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:27.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.196+0000 7f20bd6d5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20b81445d0 0x7f20b8142040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:27.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.196+0000 7f20bd6d5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b80b3240 con 0x7f20b81445d0 2023-09-12T14:56:27.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.196+0000 7f20bd6d5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b80b3520 con 0x7f20b8070840 2023-09-12T14:56:27.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.196+0000 7f20bd6d5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b813c400 con 0x7f20b80cd3a0 2023-09-12T14:56:27.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.196+0000 7f20b77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20b81445d0 0x7f20b8142040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:27.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.200+0000 7f20b77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20b81445d0 0x7f20b8142040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45168/0 (socket says 172.21.15.131:45168) 2023-09-12T14:56:27.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.200+0000 7f20b77fe640 1 -- 172.21.15.131:0/3187290234 learned_addr learned my addr 172.21.15.131:0/3187290234 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:27.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.200+0000 7f20b6ffd640 1 --2- 172.21.15.131:0/3187290234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20b8070840 0x7f20b8159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:27.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.200+0000 7f20b67fc640 1 --2- 172.21.15.131:0/3187290234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20b80cd3a0 0x7f20b80b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:27.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.200+0000 7f20b77fe640 1 -- 172.21.15.131:0/3187290234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20b80cd3a0 msgr2=0x7f20b80b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.200+0000 7f20b77fe640 1 --2- 172.21.15.131:0/3187290234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20b80cd3a0 0x7f20b80b27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.200+0000 7f20b77fe640 1 -- 172.21.15.131:0/3187290234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20b8070840 msgr2=0x7f20b8159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.200+0000 7f20b77fe640 1 --2- 172.21.15.131:0/3187290234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20b8070840 0x7f20b8159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.200+0000 7f20b77fe640 1 -- 172.21.15.131:0/3187290234 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b0009820 con 0x7f20b81445d0 2023-09-12T14:56:27.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.200+0000 7f20b77fe640 1 --2- 172.21.15.131:0/3187290234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20b81445d0 0x7f20b8142040 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f20a800c430 tx=0x7f20a800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:27.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.200+0000 7f209f7fe640 1 -- 172.21.15.131:0/3187290234 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a8005220 con 0x7f20b81445d0 2023-09-12T14:56:27.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.200+0000 7f209f7fe640 1 -- 172.21.15.131:0/3187290234 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f20a8005470 con 0x7f20b81445d0 2023-09-12T14:56:27.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.200+0000 7f20bd6d5640 1 -- 172.21.15.131:0/3187290234 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b813c6e0 con 0x7f20b81445d0 2023-09-12T14:56:27.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.200+0000 7f209f7fe640 1 -- 172.21.15.131:0/3187290234 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a8007290 con 0x7f20b81445d0 2023-09-12T14:56:27.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.200+0000 7f20bd6d5640 1 -- 172.21.15.131:0/3187290234 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b815a140 con 0x7f20b81445d0 2023-09-12T14:56:27.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.200+0000 7f20bd6d5640 1 -- 172.21.15.131:0/3187290234 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b8073580 con 0x7f20b81445d0 2023-09-12T14:56:27.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.200+0000 7f209f7fe640 1 -- 172.21.15.131:0/3187290234 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f20a80073f0 con 0x7f20b81445d0 2023-09-12T14:56:27.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.200+0000 7f209f7fe640 1 --2- 172.21.15.131:0/3187290234 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f20a4065b20 0x7f20a4067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:27.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.200+0000 7f209f7fe640 1 -- 172.21.15.131:0/3187290234 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f20a8082870 con 0x7f20b81445d0 2023-09-12T14:56:27.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.200+0000 7f20b6ffd640 1 --2- 172.21.15.131:0/3187290234 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f20a4065b20 0x7f20a4067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:27.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.200+0000 7f20b6ffd640 1 --2- 172.21.15.131:0/3187290234 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f20a4065b20 0x7f20a4067fe0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f20b00af000 tx=0x7f20b001b300 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:27.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.204+0000 7f209f7fe640 1 -- 172.21.15.131:0/3187290234 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f20a80504c0 con 0x7f20b81445d0 2023-09-12T14:56:27.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.356+0000 7f20bd6d5640 1 -- 172.21.15.131:0/3187290234 --> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f20b80abad0 con 0x7f20a4065b20 2023-09-12T14:56:27.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.360+0000 7f209f7fe640 1 -- 172.21.15.131:0/3187290234 <== mgr.4200 v2:172.21.15.131:6832/2298103651 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f20b80abad0 con 0x7f20a4065b20 2023-09-12T14:56:27.362 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 1 to deep-scrub 2023-09-12T14:56:27.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.360+0000 7f20bd6d5640 1 -- 172.21.15.131:0/3187290234 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f20a4065b20 msgr2=0x7f20a4067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.360+0000 7f20bd6d5640 1 --2- 172.21.15.131:0/3187290234 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f20a4065b20 0x7f20a4067fe0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f20b00af000 tx=0x7f20b001b300 comp rx=0 tx=0).stop 2023-09-12T14:56:27.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.360+0000 7f20bd6d5640 1 -- 172.21.15.131:0/3187290234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20b81445d0 msgr2=0x7f20b8142040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.360+0000 7f20bd6d5640 1 --2- 172.21.15.131:0/3187290234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20b81445d0 0x7f20b8142040 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f20a800c430 tx=0x7f20a800c8f0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.360+0000 7f20bd6d5640 1 -- 172.21.15.131:0/3187290234 shutdown_connections 2023-09-12T14:56:27.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.360+0000 7f20bd6d5640 1 --2- 172.21.15.131:0/3187290234 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f20a4065b20 0x7f20a4067fe0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.360+0000 7f20bd6d5640 1 --2- 172.21.15.131:0/3187290234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20b80cd3a0 0x7f20b80b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.360+0000 7f20bd6d5640 1 --2- 172.21.15.131:0/3187290234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20b81445d0 0x7f20b8142040 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.360+0000 7f20bd6d5640 1 --2- 172.21.15.131:0/3187290234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20b8070840 0x7f20b8159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.360+0000 7f20bd6d5640 1 -- 172.21.15.131:0/3187290234 >> 172.21.15.131:0/3187290234 conn(0x7f20b806a520 msgr2=0x7f20b80ce160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:27.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.360+0000 7f20bd6d5640 1 -- 172.21.15.131:0/3187290234 shutdown_connections 2023-09-12T14:56:27.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.360+0000 7f20bd6d5640 1 -- 172.21.15.131:0/3187290234 wait complete. 2023-09-12T14:56:27.380 INFO:tasks.ceph:Scrubbing osd.2 2023-09-12T14:56:27.380 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-09-12T14:56:27.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4cd0146640 1 -- 172.21.15.131:0/1048314796 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cc80b1760 msgr2=0x7f4cc815e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4cd0146640 1 --2- 172.21.15.131:0/1048314796 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cc80b1760 0x7f4cc815e820 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0009cb0 tx=0x7f4cc001cd40 comp rx=0 tx=0).stop 2023-09-12T14:56:27.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4cd0146640 1 -- 172.21.15.131:0/1048314796 shutdown_connections 2023-09-12T14:56:27.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4cd0146640 1 --2- 172.21.15.131:0/1048314796 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cc80b1760 0x7f4cc815e820 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4cd0146640 1 --2- 172.21.15.131:0/1048314796 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cc80cf2d0 0x7f4cc815e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4cd0146640 1 -- 172.21.15.131:0/1048314796 >> 172.21.15.131:0/1048314796 conn(0x7f4cc809a200 msgr2=0x7f4cc80d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:27.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4cd0146640 1 -- 172.21.15.131:0/1048314796 shutdown_connections 2023-09-12T14:56:27.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4cd0146640 1 -- 172.21.15.131:0/1048314796 wait complete. 2023-09-12T14:56:27.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4cd0146640 1 Processor -- start 2023-09-12T14:56:27.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4cd0146640 1 -- start start 2023-09-12T14:56:27.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4cd0146640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cc80b1760 0x7f4cc815b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:27.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4cd0146640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cc80cf2d0 0x7f4cc8152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:27.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4cd0146640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cc80d1680 0x7f4cc815c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:27.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4cd0146640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc815b5d0 con 0x7f4cc80b1760 2023-09-12T14:56:27.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4cd0146640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc80b3ed0 con 0x7f4cc80d1680 2023-09-12T14:56:27.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4cd0146640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc815b8b0 con 0x7f4cc80cf2d0 2023-09-12T14:56:27.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4ccdee2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cc80b1760 0x7f4cc815b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:27.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4ccdee2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cc80b1760 0x7f4cc815b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45184/0 (socket says 172.21.15.131:45184) 2023-09-12T14:56:27.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4ccdee2640 1 -- 172.21.15.131:0/2077732564 learned_addr learned my addr 172.21.15.131:0/2077732564 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:27.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4ccdee2640 1 -- 172.21.15.131:0/2077732564 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cc80cf2d0 msgr2=0x7f4cc8152480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4ccd6e1640 1 --2- 172.21.15.131:0/2077732564 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cc80cf2d0 0x7f4cc8152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:27.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4cce6e3640 1 --2- 172.21.15.131:0/2077732564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cc80d1680 0x7f4cc815c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:27.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4ccdee2640 1 --2- 172.21.15.131:0/2077732564 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cc80cf2d0 0x7f4cc8152480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4ccdee2640 1 -- 172.21.15.131:0/2077732564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cc80d1680 msgr2=0x7f4cc815c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4ccdee2640 1 --2- 172.21.15.131:0/2077732564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cc80d1680 0x7f4cc815c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4ccdee2640 1 -- 172.21.15.131:0/2077732564 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc00098e0 con 0x7f4cc80b1760 2023-09-12T14:56:27.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4ccdee2640 1 --2- 172.21.15.131:0/2077732564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cc80b1760 0x7f4cc815b090 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400c970 tx=0x7f4cc400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:27.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4cbe7fc640 1 -- 172.21.15.131:0/2077732564 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4007bb0 con 0x7f4cc80b1760 2023-09-12T14:56:27.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4cbe7fc640 1 -- 172.21.15.131:0/2077732564 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4cc4007e20 con 0x7f4cc80b1760 2023-09-12T14:56:27.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4cbe7fc640 1 -- 172.21.15.131:0/2077732564 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4005280 con 0x7f4cc80b1760 2023-09-12T14:56:27.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4cd0146640 1 -- 172.21.15.131:0/2077732564 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc815bb90 con 0x7f4cc80b1760 2023-09-12T14:56:27.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.496+0000 7f4cd0146640 1 -- 172.21.15.131:0/2077732564 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc81529c0 con 0x7f4cc80b1760 2023-09-12T14:56:27.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.500+0000 7f4cd0146640 1 -- 172.21.15.131:0/2077732564 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4cc8167f40 con 0x7f4cc80b1760 2023-09-12T14:56:27.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.500+0000 7f4cbe7fc640 1 -- 172.21.15.131:0/2077732564 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f4cc40053e0 con 0x7f4cc80b1760 2023-09-12T14:56:27.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.500+0000 7f4cbe7fc640 1 --2- 172.21.15.131:0/2077732564 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4cac065b20 0x7f4cac067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:27.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.500+0000 7f4cbe7fc640 1 -- 172.21.15.131:0/2077732564 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4cc4083ba0 con 0x7f4cc80b1760 2023-09-12T14:56:27.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.500+0000 7f4ccd6e1640 1 --2- 172.21.15.131:0/2077732564 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4cac065b20 0x7f4cac067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:27.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.500+0000 7f4cbe7fc640 1 --2- 172.21.15.131:0/2077732564 >> [v2:172.21.15.131:6824/3891085905,v1:172.21.15.131:6825/3891085905] conn(0x7f4cac06b980 0x7f4cac06dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:27.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.500+0000 7f4cbe7fc640 1 -- 172.21.15.131:0/2077732564 --> [v2:172.21.15.131:6824/3891085905,v1:172.21.15.131:6825/3891085905] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4cac06e450 con 0x7f4cac06b980 2023-09-12T14:56:27.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.500+0000 7f4cbe7fc640 1 -- 172.21.15.131:0/2077732564 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cc4083ff0 con 0x7f4cc80b1760 2023-09-12T14:56:27.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.500+0000 7f4cce6e3640 1 --2- 172.21.15.131:0/2077732564 >> [v2:172.21.15.131:6824/3891085905,v1:172.21.15.131:6825/3891085905] conn(0x7f4cac06b980 0x7f4cac06dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:27.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.500+0000 7f4ccd6e1640 1 --2- 172.21.15.131:0/2077732564 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4cac065b20 0x7f4cac067fe0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4cc81f75f0 tx=0x7f4cb8008040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:27.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.500+0000 7f4cce6e3640 1 --2- 172.21.15.131:0/2077732564 >> [v2:172.21.15.131:6824/3891085905,v1:172.21.15.131:6825/3891085905] conn(0x7f4cac06b980 0x7f4cac06dda0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:27.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.500+0000 7f4cbe7fc640 1 -- 172.21.15.131:0/2077732564 <== osd.2 v2:172.21.15.131:6824/3891085905 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f4cac06e450 con 0x7f4cac06b980 2023-09-12T14:56:27.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.512+0000 7f4cd0146640 1 -- 172.21.15.131:0/2077732564 --> [v2:172.21.15.131:6824/3891085905,v1:172.21.15.131:6825/3891085905] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4cc8073580 con 0x7f4cac06b980 2023-09-12T14:56:27.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.512+0000 7f4cbe7fc640 1 -- 172.21.15.131:0/2077732564 <== osd.2 v2:172.21.15.131:6824/3891085905 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f4cc8073580 con 0x7f4cac06b980 2023-09-12T14:56:27.522 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-09-12T14:56:27.522 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-12T14:56:27.522 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-09-12T14:56:27.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.512+0000 7f4cd0146640 1 -- 172.21.15.131:0/2077732564 >> [v2:172.21.15.131:6824/3891085905,v1:172.21.15.131:6825/3891085905] conn(0x7f4cac06b980 msgr2=0x7f4cac06dda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.512+0000 7f4cd0146640 1 --2- 172.21.15.131:0/2077732564 >> [v2:172.21.15.131:6824/3891085905,v1:172.21.15.131:6825/3891085905] conn(0x7f4cac06b980 0x7f4cac06dda0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.512+0000 7f4cd0146640 1 -- 172.21.15.131:0/2077732564 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4cac065b20 msgr2=0x7f4cac067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.512+0000 7f4cd0146640 1 --2- 172.21.15.131:0/2077732564 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4cac065b20 0x7f4cac067fe0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4cc81f75f0 tx=0x7f4cb8008040 comp rx=0 tx=0).stop 2023-09-12T14:56:27.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.512+0000 7f4cd0146640 1 -- 172.21.15.131:0/2077732564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cc80b1760 msgr2=0x7f4cc815b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.512+0000 7f4cd0146640 1 --2- 172.21.15.131:0/2077732564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cc80b1760 0x7f4cc815b090 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400c970 tx=0x7f4cc400ce30 comp rx=0 tx=0).stop 2023-09-12T14:56:27.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.512+0000 7f4cce6e3640 1 -- 172.21.15.131:0/2077732564 reap_dead start 2023-09-12T14:56:27.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.512+0000 7f4cd0146640 1 -- 172.21.15.131:0/2077732564 shutdown_connections 2023-09-12T14:56:27.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.512+0000 7f4cd0146640 1 -- 172.21.15.131:0/2077732564 >> 172.21.15.131:0/2077732564 conn(0x7f4cc809a200 msgr2=0x7f4cc806abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:27.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.512+0000 7f4cd0146640 1 -- 172.21.15.131:0/2077732564 shutdown_connections 2023-09-12T14:56:27.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.512+0000 7f4cd0146640 1 -- 172.21.15.131:0/2077732564 wait complete. 2023-09-12T14:56:27.531 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-09-12T14:56:27.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.640+0000 7fe366ab4640 1 -- 172.21.15.131:0/3039668134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3600b1760 msgr2=0x7fe36015e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.640+0000 7fe366ab4640 1 --2- 172.21.15.131:0/3039668134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3600b1760 0x7fe36015e820 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe348009870 tx=0x7fe34801c800 comp rx=0 tx=0).stop 2023-09-12T14:56:27.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.640+0000 7fe366ab4640 1 -- 172.21.15.131:0/3039668134 shutdown_connections 2023-09-12T14:56:27.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.640+0000 7fe366ab4640 1 --2- 172.21.15.131:0/3039668134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3600b1760 0x7fe36015e820 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.640+0000 7fe366ab4640 1 --2- 172.21.15.131:0/3039668134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3600cf2d0 0x7fe36015e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.640+0000 7fe366ab4640 1 -- 172.21.15.131:0/3039668134 >> 172.21.15.131:0/3039668134 conn(0x7fe36009a200 msgr2=0x7fe3600d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:27.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.640+0000 7fe366ab4640 1 -- 172.21.15.131:0/3039668134 shutdown_connections 2023-09-12T14:56:27.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.644+0000 7fe366ab4640 1 -- 172.21.15.131:0/3039668134 wait complete. 2023-09-12T14:56:27.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.644+0000 7fe366ab4640 1 Processor -- start 2023-09-12T14:56:27.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.644+0000 7fe366ab4640 1 -- start start 2023-09-12T14:56:27.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.644+0000 7fe366ab4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3600b1760 0x7fe360159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:27.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.644+0000 7fe366ab4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3600cf2d0 0x7fe36014cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:27.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.644+0000 7fe366ab4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3600d1680 0x7fe3600b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:27.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.644+0000 7fe366ab4640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36015ccd0 con 0x7fe3600d1680 2023-09-12T14:56:27.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.644+0000 7fe366ab4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36015c430 con 0x7fe3600b1760 2023-09-12T14:56:27.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.644+0000 7fe366ab4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36015c710 con 0x7fe3600cf2d0 2023-09-12T14:56:27.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.644+0000 7fe365051640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3600d1680 0x7fe3600b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:27.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.644+0000 7fe365051640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3600d1680 0x7fe3600b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45204/0 (socket says 172.21.15.131:45204) 2023-09-12T14:56:27.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.644+0000 7fe365051640 1 -- 172.21.15.131:0/3103944723 learned_addr learned my addr 172.21.15.131:0/3103944723 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:27.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.644+0000 7fe365051640 1 -- 172.21.15.131:0/3103944723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3600cf2d0 msgr2=0x7fe36014cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.644+0000 7fe365051640 1 --2- 172.21.15.131:0/3103944723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3600cf2d0 0x7fe36014cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.644+0000 7fe365051640 1 -- 172.21.15.131:0/3103944723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3600b1760 msgr2=0x7fe360159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.644+0000 7fe364850640 1 --2- 172.21.15.131:0/3103944723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3600b1760 0x7fe360159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:27.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.644+0000 7fe365051640 1 --2- 172.21.15.131:0/3103944723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3600b1760 0x7fe360159ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.644+0000 7fe365051640 1 -- 172.21.15.131:0/3103944723 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3480094a0 con 0x7fe3600d1680 2023-09-12T14:56:27.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.644+0000 7fe365051640 1 --2- 172.21.15.131:0/3103944723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3600d1680 0x7fe3600b3b60 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe34801a430 tx=0x7fe3480a5770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:27.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.644+0000 7fe35d7fa640 1 -- 172.21.15.131:0/3103944723 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe348005230 con 0x7fe3600d1680 2023-09-12T14:56:27.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.644+0000 7fe35d7fa640 1 -- 172.21.15.131:0/3103944723 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe348005480 con 0x7fe3600d1680 2023-09-12T14:56:27.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.644+0000 7fe366ab4640 1 -- 172.21.15.131:0/3103944723 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36015c9f0 con 0x7fe3600d1680 2023-09-12T14:56:27.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.644+0000 7fe35d7fa640 1 -- 172.21.15.131:0/3103944723 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe348004930 con 0x7fe3600d1680 2023-09-12T14:56:27.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.644+0000 7fe366ab4640 1 -- 172.21.15.131:0/3103944723 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36015c0e0 con 0x7fe3600d1680 2023-09-12T14:56:27.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.644+0000 7fe366ab4640 1 -- 172.21.15.131:0/3103944723 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe360073580 con 0x7fe3600d1680 2023-09-12T14:56:27.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.648+0000 7fe35d7fa640 1 -- 172.21.15.131:0/3103944723 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fe3480b4040 con 0x7fe3600d1680 2023-09-12T14:56:27.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.648+0000 7fe35d7fa640 1 --2- 172.21.15.131:0/3103944723 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe344065b20 0x7fe344067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:27.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.648+0000 7fe35d7fa640 1 -- 172.21.15.131:0/3103944723 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe34811d0d0 con 0x7fe3600d1680 2023-09-12T14:56:27.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.648+0000 7fe364850640 1 --2- 172.21.15.131:0/3103944723 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe344065b20 0x7fe344067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:27.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.648+0000 7fe364850640 1 --2- 172.21.15.131:0/3103944723 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe344065b20 0x7fe344067fe0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe36014d8c0 tx=0x7fe350005cf0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:27.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.652+0000 7fe35d7fa640 1 -- 172.21.15.131:0/3103944723 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fe3480eadb0 con 0x7fe3600d1680 2023-09-12T14:56:27.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.804+0000 7fe366ab4640 1 -- 172.21.15.131:0/3103944723 --> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe360167f40 con 0x7fe344065b20 2023-09-12T14:56:27.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.804+0000 7fe35d7fa640 1 -- 172.21.15.131:0/3103944723 <== mgr.4200 v2:172.21.15.131:6832/2298103651 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe35000da80 con 0x7fe344065b20 2023-09-12T14:56:27.806 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 2 to deep-scrub 2023-09-12T14:56:27.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.804+0000 7fe366ab4640 1 -- 172.21.15.131:0/3103944723 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe344065b20 msgr2=0x7fe344067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.804+0000 7fe366ab4640 1 --2- 172.21.15.131:0/3103944723 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe344065b20 0x7fe344067fe0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe36014d8c0 tx=0x7fe350005cf0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.804+0000 7fe366ab4640 1 -- 172.21.15.131:0/3103944723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3600d1680 msgr2=0x7fe3600b3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.804+0000 7fe366ab4640 1 --2- 172.21.15.131:0/3103944723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3600d1680 0x7fe3600b3b60 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe34801a430 tx=0x7fe3480a5770 comp rx=0 tx=0).stop 2023-09-12T14:56:27.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.804+0000 7fe366ab4640 1 -- 172.21.15.131:0/3103944723 shutdown_connections 2023-09-12T14:56:27.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.804+0000 7fe366ab4640 1 --2- 172.21.15.131:0/3103944723 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe344065b20 0x7fe344067fe0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.804+0000 7fe366ab4640 1 --2- 172.21.15.131:0/3103944723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3600cf2d0 0x7fe36014cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.804+0000 7fe366ab4640 1 --2- 172.21.15.131:0/3103944723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3600d1680 0x7fe3600b3b60 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.804+0000 7fe366ab4640 1 --2- 172.21.15.131:0/3103944723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3600b1760 0x7fe360159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.804+0000 7fe366ab4640 1 -- 172.21.15.131:0/3103944723 >> 172.21.15.131:0/3103944723 conn(0x7fe36009a200 msgr2=0x7fe36015e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:27.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.804+0000 7fe366ab4640 1 -- 172.21.15.131:0/3103944723 shutdown_connections 2023-09-12T14:56:27.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.804+0000 7fe366ab4640 1 -- 172.21.15.131:0/3103944723 wait complete. 2023-09-12T14:56:27.824 INFO:tasks.ceph:Scrubbing osd.3 2023-09-12T14:56:27.825 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-09-12T14:56:27.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.952+0000 7f0ddb2a2640 1 -- 172.21.15.131:0/3066859162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd406c060 msgr2=0x7f0dd41626d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.952+0000 7f0ddb2a2640 1 --2- 172.21.15.131:0/3066859162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd406c060 0x7f0dd41626d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc009bc0 tx=0x7f0dcc01ca80 comp rx=0 tx=0).stop 2023-09-12T14:56:27.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.952+0000 7f0ddb2a2640 1 -- 172.21.15.131:0/3066859162 shutdown_connections 2023-09-12T14:56:27.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.952+0000 7f0ddb2a2640 1 --2- 172.21.15.131:0/3066859162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd406c060 0x7f0dd41626d0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.952+0000 7f0ddb2a2640 1 -- 172.21.15.131:0/3066859162 >> 172.21.15.131:0/3066859162 conn(0x7f0dd4069840 msgr2=0x7f0dd40cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:27.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.952+0000 7f0ddb2a2640 1 -- 172.21.15.131:0/3066859162 shutdown_connections 2023-09-12T14:56:27.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.952+0000 7f0ddb2a2640 1 -- 172.21.15.131:0/3066859162 wait complete. 2023-09-12T14:56:27.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.956+0000 7f0ddb2a2640 1 Processor -- start 2023-09-12T14:56:27.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.956+0000 7f0ddb2a2640 1 -- start start 2023-09-12T14:56:27.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.956+0000 7f0ddb2a2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd406c060 0x7f0dd4148b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:27.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.956+0000 7f0ddb2a2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0dd40ab730 0x7f0dd40b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:27.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.956+0000 7f0ddb2a2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0dd4158180 0x7f0dd4143ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:27.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.956+0000 7f0ddb2a2640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd41494d0 con 0x7f0dd406c060 2023-09-12T14:56:27.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.956+0000 7f0ddb2a2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd41497b0 con 0x7f0dd4158180 2023-09-12T14:56:27.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.956+0000 7f0ddb2a2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd4149a90 con 0x7f0dd40ab730 2023-09-12T14:56:27.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.956+0000 7f0dd903e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd406c060 0x7f0dd4148b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:27.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.956+0000 7f0dd903e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd406c060 0x7f0dd4148b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45224/0 (socket says 172.21.15.131:45224) 2023-09-12T14:56:27.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.956+0000 7f0dd903e640 1 -- 172.21.15.131:0/441105755 learned_addr learned my addr 172.21.15.131:0/441105755 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:27.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.956+0000 7f0dd983f640 1 --2- 172.21.15.131:0/441105755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0dd4158180 0x7f0dd4143ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:27.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.956+0000 7f0dd903e640 1 -- 172.21.15.131:0/441105755 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0dd40ab730 msgr2=0x7f0dd40b14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.956+0000 7f0dd883d640 1 --2- 172.21.15.131:0/441105755 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0dd40ab730 0x7f0dd40b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:27.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.956+0000 7f0dd903e640 1 --2- 172.21.15.131:0/441105755 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0dd40ab730 0x7f0dd40b14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.956+0000 7f0dd903e640 1 -- 172.21.15.131:0/441105755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0dd4158180 msgr2=0x7f0dd4143ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.956+0000 7f0dd903e640 1 --2- 172.21.15.131:0/441105755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0dd4158180 0x7f0dd4143ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.956+0000 7f0dd903e640 1 -- 172.21.15.131:0/441105755 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dcc009820 con 0x7f0dd406c060 2023-09-12T14:56:27.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.956+0000 7f0dd983f640 1 --2- 172.21.15.131:0/441105755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0dd4158180 0x7f0dd4143ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:27.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.956+0000 7f0dd903e640 1 --2- 172.21.15.131:0/441105755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd406c060 0x7f0dd4148b60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0dd000c420 tx=0x7f0dd000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:27.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.956+0000 7f0dc9ffb640 1 -- 172.21.15.131:0/441105755 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd0004230 con 0x7f0dd406c060 2023-09-12T14:56:27.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.956+0000 7f0dc9ffb640 1 -- 172.21.15.131:0/441105755 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0dd00044a0 con 0x7f0dd406c060 2023-09-12T14:56:27.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.956+0000 7f0dc9ffb640 1 -- 172.21.15.131:0/441105755 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd0005260 con 0x7f0dd406c060 2023-09-12T14:56:27.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.956+0000 7f0ddb2a2640 1 -- 172.21.15.131:0/441105755 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd4149d70 con 0x7f0dd406c060 2023-09-12T14:56:27.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.956+0000 7f0ddb2a2640 1 -- 172.21.15.131:0/441105755 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd41490a0 con 0x7f0dd406c060 2023-09-12T14:56:27.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.956+0000 7f0ddb2a2640 1 -- 172.21.15.131:0/441105755 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0dd4073580 con 0x7f0dd406c060 2023-09-12T14:56:27.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.960+0000 7f0dc9ffb640 1 -- 172.21.15.131:0/441105755 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f0dd00053c0 con 0x7f0dd406c060 2023-09-12T14:56:27.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.960+0000 7f0dc9ffb640 1 --2- 172.21.15.131:0/441105755 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0db8065b20 0x7f0db8067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:27.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.960+0000 7f0dc9ffb640 1 -- 172.21.15.131:0/441105755 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0dd0083190 con 0x7f0dd406c060 2023-09-12T14:56:27.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.960+0000 7f0dd883d640 1 --2- 172.21.15.131:0/441105755 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0db8065b20 0x7f0db8067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:27.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.960+0000 7f0dc9ffb640 1 --2- 172.21.15.131:0/441105755 >> [v2:172.21.15.131:6816/2354456273,v1:172.21.15.131:6817/2354456273] conn(0x7f0db806b980 0x7f0db806dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:27.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.960+0000 7f0dc9ffb640 1 -- 172.21.15.131:0/441105755 --> [v2:172.21.15.131:6816/2354456273,v1:172.21.15.131:6817/2354456273] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0db806e450 con 0x7f0db806b980 2023-09-12T14:56:27.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.960+0000 7f0dc9ffb640 1 -- 172.21.15.131:0/441105755 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0dd00835e0 con 0x7f0dd406c060 2023-09-12T14:56:27.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.960+0000 7f0dd983f640 1 --2- 172.21.15.131:0/441105755 >> [v2:172.21.15.131:6816/2354456273,v1:172.21.15.131:6817/2354456273] conn(0x7f0db806b980 0x7f0db806dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:27.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.960+0000 7f0dd883d640 1 --2- 172.21.15.131:0/441105755 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0db8065b20 0x7f0db8067fe0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0dd414f870 tx=0x7f0dc4008040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:27.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.960+0000 7f0dd983f640 1 --2- 172.21.15.131:0/441105755 >> [v2:172.21.15.131:6816/2354456273,v1:172.21.15.131:6817/2354456273] conn(0x7f0db806b980 0x7f0db806dda0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:27.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.960+0000 7f0dc9ffb640 1 -- 172.21.15.131:0/441105755 <== osd.3 v2:172.21.15.131:6816/2354456273 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f0db806e450 con 0x7f0db806b980 2023-09-12T14:56:27.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.972+0000 7f0ddb2a2640 1 -- 172.21.15.131:0/441105755 --> [v2:172.21.15.131:6816/2354456273,v1:172.21.15.131:6817/2354456273] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0dd414a220 con 0x7f0db806b980 2023-09-12T14:56:27.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.972+0000 7f0dc9ffb640 1 -- 172.21.15.131:0/441105755 <== osd.3 v2:172.21.15.131:6816/2354456273 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f0dd414a220 con 0x7f0db806b980 2023-09-12T14:56:27.976 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-09-12T14:56:27.976 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-12T14:56:27.976 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-09-12T14:56:27.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.972+0000 7f0ddb2a2640 1 -- 172.21.15.131:0/441105755 >> [v2:172.21.15.131:6816/2354456273,v1:172.21.15.131:6817/2354456273] conn(0x7f0db806b980 msgr2=0x7f0db806dda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.972+0000 7f0ddb2a2640 1 --2- 172.21.15.131:0/441105755 >> [v2:172.21.15.131:6816/2354456273,v1:172.21.15.131:6817/2354456273] conn(0x7f0db806b980 0x7f0db806dda0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.972+0000 7f0ddb2a2640 1 -- 172.21.15.131:0/441105755 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0db8065b20 msgr2=0x7f0db8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.972+0000 7f0ddb2a2640 1 --2- 172.21.15.131:0/441105755 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f0db8065b20 0x7f0db8067fe0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0dd414f870 tx=0x7f0dc4008040 comp rx=0 tx=0).stop 2023-09-12T14:56:27.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.972+0000 7f0ddb2a2640 1 -- 172.21.15.131:0/441105755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd406c060 msgr2=0x7f0dd4148b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:27.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.972+0000 7f0ddb2a2640 1 --2- 172.21.15.131:0/441105755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd406c060 0x7f0dd4148b60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0dd000c420 tx=0x7f0dd000c8e0 comp rx=0 tx=0).stop 2023-09-12T14:56:27.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.972+0000 7f0dd983f640 1 -- 172.21.15.131:0/441105755 reap_dead start 2023-09-12T14:56:27.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.972+0000 7f0ddb2a2640 1 -- 172.21.15.131:0/441105755 shutdown_connections 2023-09-12T14:56:27.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.972+0000 7f0ddb2a2640 1 -- 172.21.15.131:0/441105755 >> 172.21.15.131:0/441105755 conn(0x7f0dd4069840 msgr2=0x7f0dd406eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:27.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.972+0000 7f0ddb2a2640 1 -- 172.21.15.131:0/441105755 shutdown_connections 2023-09-12T14:56:27.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:27.972+0000 7f0ddb2a2640 1 -- 172.21.15.131:0/441105755 wait complete. 2023-09-12T14:56:27.991 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-09-12T14:56:28.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.100+0000 7f72cdc8a640 1 -- 172.21.15.131:0/3358952923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72b40047d0 msgr2=0x7f72b4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:28.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.100+0000 7f72cdc8a640 1 --2- 172.21.15.131:0/3358952923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72b40047d0 0x7f72b4004c50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f72c80b0560 tx=0x7f72b801f9e0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.100+0000 7f72c7fff640 1 -- 172.21.15.131:0/3358952923 reap_dead start 2023-09-12T14:56:28.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.100+0000 7f72cdc8a640 1 -- 172.21.15.131:0/3358952923 shutdown_connections 2023-09-12T14:56:28.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.100+0000 7f72cdc8a640 1 -- 172.21.15.131:0/3358952923 >> 172.21.15.131:0/3358952923 conn(0x7f72c80ceac0 msgr2=0x7f72c80ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:28.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.100+0000 7f72cdc8a640 1 -- 172.21.15.131:0/3358952923 shutdown_connections 2023-09-12T14:56:28.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.100+0000 7f72cdc8a640 1 -- 172.21.15.131:0/3358952923 wait complete. 2023-09-12T14:56:28.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.100+0000 7f72cdc8a640 1 Processor -- start 2023-09-12T14:56:28.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.100+0000 7f72cdc8a640 1 -- start start 2023-09-12T14:56:28.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72cdc8a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b40047d0 0x7f72c80c4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:28.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72cdc8a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72c80b2070 0x7f72c80c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:28.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72cdc8a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72c806a350 0x7f72c806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:28.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72cdc8a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c80c1900 con 0x7f72c806a350 2023-09-12T14:56:28.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72cdc8a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c80c89b0 con 0x7f72b40047d0 2023-09-12T14:56:28.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72cdc8a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c80c51e0 con 0x7f72c80b2070 2023-09-12T14:56:28.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72c7fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72c806a350 0x7f72c806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:28.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72c7fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72c806a350 0x7f72c806a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45246/0 (socket says 172.21.15.131:45246) 2023-09-12T14:56:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72c7fff640 1 -- 172.21.15.131:0/2335886043 learned_addr learned my addr 172.21.15.131:0/2335886043 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72c77fe640 1 --2- 172.21.15.131:0/2335886043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b40047d0 0x7f72c80c4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72c6ffd640 1 --2- 172.21.15.131:0/2335886043 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72c80b2070 0x7f72c80c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72c7fff640 1 -- 172.21.15.131:0/2335886043 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72c80b2070 msgr2=0x7f72c80c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:28.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72c7fff640 1 --2- 172.21.15.131:0/2335886043 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72c80b2070 0x7f72c80c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72c7fff640 1 -- 172.21.15.131:0/2335886043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b40047d0 msgr2=0x7f72c80c4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:28.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72c7fff640 1 --2- 172.21.15.131:0/2335886043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b40047d0 0x7f72c80c4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72c7fff640 1 -- 172.21.15.131:0/2335886043 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c80b24b0 con 0x7f72c806a350 2023-09-12T14:56:28.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72c7fff640 1 --2- 172.21.15.131:0/2335886043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72c806a350 0x7f72c806a7d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f72c80cc8d0 tx=0x7f72c000d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:28.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72bffff640 1 -- 172.21.15.131:0/2335886043 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c0004230 con 0x7f72c806a350 2023-09-12T14:56:28.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72bffff640 1 -- 172.21.15.131:0/2335886043 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f72c00044a0 con 0x7f72c806a350 2023-09-12T14:56:28.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72cdc8a640 1 -- 172.21.15.131:0/2335886043 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c80c2e00 con 0x7f72c806a350 2023-09-12T14:56:28.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72bffff640 1 -- 172.21.15.131:0/2335886043 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c000f8f0 con 0x7f72c806a350 2023-09-12T14:56:28.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72cdc8a640 1 -- 172.21.15.131:0/2335886043 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c80c33c0 con 0x7f72c806a350 2023-09-12T14:56:28.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72cdc8a640 1 -- 172.21.15.131:0/2335886043 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c8073580 con 0x7f72c806a350 2023-09-12T14:56:28.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72bffff640 1 -- 172.21.15.131:0/2335886043 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f72c000fa50 con 0x7f72c806a350 2023-09-12T14:56:28.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72bffff640 1 --2- 172.21.15.131:0/2335886043 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f72a8065b50 0x7f72a8068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:28.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72bffff640 1 -- 172.21.15.131:0/2335886043 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f72c00826c0 con 0x7f72c806a350 2023-09-12T14:56:28.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72c77fe640 1 --2- 172.21.15.131:0/2335886043 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f72a8065b50 0x7f72a8068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:28.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.104+0000 7f72c77fe640 1 --2- 172.21.15.131:0/2335886043 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f72a8065b50 0x7f72a8068010 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009b70 tx=0x7f72b0005750 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:28.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.108+0000 7f72bffff640 1 -- 172.21.15.131:0/2335886043 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f72c00503a0 con 0x7f72c806a350 2023-09-12T14:56:28.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.252+0000 7f72cdc8a640 1 -- 172.21.15.131:0/2335886043 --> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f72c80aba50 con 0x7f72a8065b50 2023-09-12T14:56:28.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.252+0000 7f72bffff640 1 -- 172.21.15.131:0/2335886043 <== mgr.4200 v2:172.21.15.131:6832/2298103651 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f72b0016070 con 0x7f72a8065b50 2023-09-12T14:56:28.254 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 3 to deep-scrub 2023-09-12T14:56:28.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.252+0000 7f72cdc8a640 1 -- 172.21.15.131:0/2335886043 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f72a8065b50 msgr2=0x7f72a8068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:28.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.252+0000 7f72cdc8a640 1 --2- 172.21.15.131:0/2335886043 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f72a8065b50 0x7f72a8068010 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009b70 tx=0x7f72b0005750 comp rx=0 tx=0).stop 2023-09-12T14:56:28.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.252+0000 7f72cdc8a640 1 -- 172.21.15.131:0/2335886043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72c806a350 msgr2=0x7f72c806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:28.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.252+0000 7f72cdc8a640 1 --2- 172.21.15.131:0/2335886043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72c806a350 0x7f72c806a7d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f72c80cc8d0 tx=0x7f72c000d9a0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.252+0000 7f72cdc8a640 1 -- 172.21.15.131:0/2335886043 shutdown_connections 2023-09-12T14:56:28.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.252+0000 7f72cdc8a640 1 --2- 172.21.15.131:0/2335886043 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f72a8065b50 0x7f72a8068010 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.252+0000 7f72cdc8a640 1 --2- 172.21.15.131:0/2335886043 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72c80b2070 0x7f72c80c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.252+0000 7f72cdc8a640 1 --2- 172.21.15.131:0/2335886043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72c806a350 0x7f72c806a7d0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.252+0000 7f72cdc8a640 1 --2- 172.21.15.131:0/2335886043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b40047d0 0x7f72c80c4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.252+0000 7f72cdc8a640 1 -- 172.21.15.131:0/2335886043 >> 172.21.15.131:0/2335886043 conn(0x7f72c80ceac0 msgr2=0x7f72c8159920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:28.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.252+0000 7f72cdc8a640 1 -- 172.21.15.131:0/2335886043 shutdown_connections 2023-09-12T14:56:28.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.252+0000 7f72cdc8a640 1 -- 172.21.15.131:0/2335886043 wait complete. 2023-09-12T14:56:28.271 INFO:tasks.ceph:Scrubbing osd.4 2023-09-12T14:56:28.272 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-09-12T14:56:28.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.400+0000 7f6a183e3640 1 -- 172.21.15.131:0/3477267735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a1006b050 msgr2=0x7f6a1006b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:28.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.400+0000 7f6a183e3640 1 --2- 172.21.15.131:0/3477267735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a1006b050 0x7f6a1006b4e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f6a08009840 tx=0x7f6a0801c7d0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.400+0000 7f6a183e3640 1 -- 172.21.15.131:0/3477267735 shutdown_connections 2023-09-12T14:56:28.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.400+0000 7f6a183e3640 1 --2- 172.21.15.131:0/3477267735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a1006b050 0x7f6a1006b4e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.400+0000 7f6a183e3640 1 --2- 172.21.15.131:0/3477267735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a100ab730 0x7f6a100abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.405 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.400+0000 7f6a183e3640 1 -- 172.21.15.131:0/3477267735 >> 172.21.15.131:0/3477267735 conn(0x7f6a10069840 msgr2=0x7f6a100cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:28.405 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.400+0000 7f6a183e3640 1 -- 172.21.15.131:0/3477267735 shutdown_connections 2023-09-12T14:56:28.405 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.400+0000 7f6a183e3640 1 -- 172.21.15.131:0/3477267735 wait complete. 2023-09-12T14:56:28.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.400+0000 7f6a183e3640 1 Processor -- start 2023-09-12T14:56:28.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.400+0000 7f6a183e3640 1 -- start start 2023-09-12T14:56:28.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a183e3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a1006b050 0x7f6a10145c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:28.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a183e3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a100ab730 0x7f6a101463c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:28.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a183e3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a10158180 0x7f6a10156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:28.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a183e3640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a101461a0 con 0x7f6a100ab730 2023-09-12T14:56:28.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a183e3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a10146d30 con 0x7f6a1006b050 2023-09-12T14:56:28.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a183e3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a10147010 con 0x7f6a10158180 2023-09-12T14:56:28.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a1597e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a100ab730 0x7f6a101463c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:28.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a1597e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a100ab730 0x7f6a101463c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40464/0 (socket says 172.21.15.131:40464) 2023-09-12T14:56:28.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a1597e640 1 -- 172.21.15.131:0/2091286499 learned_addr learned my addr 172.21.15.131:0/2091286499 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:28.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a16980640 1 --2- 172.21.15.131:0/2091286499 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a10158180 0x7f6a10156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:28.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a1617f640 1 --2- 172.21.15.131:0/2091286499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a1006b050 0x7f6a10145c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:28.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a1597e640 1 -- 172.21.15.131:0/2091286499 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a10158180 msgr2=0x7f6a10156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:28.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a1597e640 1 --2- 172.21.15.131:0/2091286499 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a10158180 0x7f6a10156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a1597e640 1 -- 172.21.15.131:0/2091286499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a1006b050 msgr2=0x7f6a10145c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:28.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a1597e640 1 --2- 172.21.15.131:0/2091286499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a1006b050 0x7f6a10145c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a1597e640 1 -- 172.21.15.131:0/2091286499 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a080094a0 con 0x7f6a100ab730 2023-09-12T14:56:28.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a1597e640 1 --2- 172.21.15.131:0/2091286499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a100ab730 0x7f6a101463c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400c410 tx=0x7f6a0400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:28.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a0effd640 1 -- 172.21.15.131:0/2091286499 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a04011790 con 0x7f6a100ab730 2023-09-12T14:56:28.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a0effd640 1 -- 172.21.15.131:0/2091286499 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6a040119e0 con 0x7f6a100ab730 2023-09-12T14:56:28.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a0effd640 1 -- 172.21.15.131:0/2091286499 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a04011dd0 con 0x7f6a100ab730 2023-09-12T14:56:28.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a183e3640 1 -- 172.21.15.131:0/2091286499 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a101472f0 con 0x7f6a100ab730 2023-09-12T14:56:28.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a183e3640 1 -- 172.21.15.131:0/2091286499 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a10159240 con 0x7f6a100ab730 2023-09-12T14:56:28.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a183e3640 1 -- 172.21.15.131:0/2091286499 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6a10073580 con 0x7f6a100ab730 2023-09-12T14:56:28.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a0effd640 1 -- 172.21.15.131:0/2091286499 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f6a0401c460 con 0x7f6a100ab730 2023-09-12T14:56:28.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a0effd640 1 --2- 172.21.15.131:0/2091286499 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f69fc065b20 0x7f69fc067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:28.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a0effd640 1 -- 172.21.15.131:0/2091286499 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6a04083c90 con 0x7f6a100ab730 2023-09-12T14:56:28.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a1617f640 1 --2- 172.21.15.131:0/2091286499 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f69fc065b20 0x7f69fc067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:28.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a0effd640 1 --2- 172.21.15.131:0/2091286499 >> [v2:172.21.15.187:6800/1649621344,v1:172.21.15.187:6802/1649621344] conn(0x7f69fc06b980 0x7f69fc06dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:28.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a0effd640 1 -- 172.21.15.131:0/2091286499 --> [v2:172.21.15.187:6800/1649621344,v1:172.21.15.187:6802/1649621344] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f69fc06e450 con 0x7f69fc06b980 2023-09-12T14:56:28.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a0effd640 1 -- 172.21.15.131:0/2091286499 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a040840e0 con 0x7f6a100ab730 2023-09-12T14:56:28.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a1617f640 1 --2- 172.21.15.131:0/2091286499 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f69fc065b20 0x7f69fc067fe0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6a00005c50 tx=0x7f6a00005be0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:28.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.404+0000 7f6a16980640 1 --2- 172.21.15.131:0/2091286499 >> [v2:172.21.15.187:6800/1649621344,v1:172.21.15.187:6802/1649621344] conn(0x7f69fc06b980 0x7f69fc06dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:28.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.408+0000 7f6a16980640 1 --2- 172.21.15.131:0/2091286499 >> [v2:172.21.15.187:6800/1649621344,v1:172.21.15.187:6802/1649621344] conn(0x7f69fc06b980 0x7f69fc06dda0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:28.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.408+0000 7f6a0effd640 1 -- 172.21.15.131:0/2091286499 <== osd.4 v2:172.21.15.187:6800/1649621344 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f69fc06e450 con 0x7f69fc06b980 2023-09-12T14:56:28.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.420+0000 7f6a183e3640 1 -- 172.21.15.131:0/2091286499 --> [v2:172.21.15.187:6800/1649621344,v1:172.21.15.187:6802/1649621344] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6a1006ec30 con 0x7f69fc06b980 2023-09-12T14:56:28.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.420+0000 7f6a0effd640 1 -- 172.21.15.131:0/2091286499 <== osd.4 v2:172.21.15.187:6800/1649621344 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f6a1006ec30 con 0x7f69fc06b980 2023-09-12T14:56:28.424 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-09-12T14:56:28.425 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-12T14:56:28.425 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-09-12T14:56:28.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.420+0000 7f6a183e3640 1 -- 172.21.15.131:0/2091286499 >> [v2:172.21.15.187:6800/1649621344,v1:172.21.15.187:6802/1649621344] conn(0x7f69fc06b980 msgr2=0x7f69fc06dda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:28.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.420+0000 7f6a183e3640 1 --2- 172.21.15.131:0/2091286499 >> [v2:172.21.15.187:6800/1649621344,v1:172.21.15.187:6802/1649621344] conn(0x7f69fc06b980 0x7f69fc06dda0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.420+0000 7f6a183e3640 1 -- 172.21.15.131:0/2091286499 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f69fc065b20 msgr2=0x7f69fc067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:28.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.420+0000 7f6a183e3640 1 --2- 172.21.15.131:0/2091286499 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f69fc065b20 0x7f69fc067fe0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6a00005c50 tx=0x7f6a00005be0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.420+0000 7f6a183e3640 1 -- 172.21.15.131:0/2091286499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a100ab730 msgr2=0x7f6a101463c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:28.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.420+0000 7f6a183e3640 1 --2- 172.21.15.131:0/2091286499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a100ab730 0x7f6a101463c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400c410 tx=0x7f6a0400c8d0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.420+0000 7f6a16980640 1 -- 172.21.15.131:0/2091286499 reap_dead start 2023-09-12T14:56:28.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.420+0000 7f6a183e3640 1 -- 172.21.15.131:0/2091286499 shutdown_connections 2023-09-12T14:56:28.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.420+0000 7f6a183e3640 1 -- 172.21.15.131:0/2091286499 >> 172.21.15.131:0/2091286499 conn(0x7f6a10069840 msgr2=0x7f6a100ae690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:28.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.420+0000 7f6a183e3640 1 -- 172.21.15.131:0/2091286499 shutdown_connections 2023-09-12T14:56:28.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.424+0000 7f6a183e3640 1 -- 172.21.15.131:0/2091286499 wait complete. 2023-09-12T14:56:28.440 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-09-12T14:56:28.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.556+0000 7fe32b39c640 1 -- 172.21.15.131:0/3355731956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe30c005190 msgr2=0x7fe30c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:28.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.556+0000 7fe32b39c640 1 --2- 172.21.15.131:0/3355731956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe30c005190 0x7fe30c00b990 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe31400c610 tx=0x7fe31401fa80 comp rx=0 tx=0).stop 2023-09-12T14:56:28.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.556+0000 7fe32b39c640 1 -- 172.21.15.131:0/3355731956 shutdown_connections 2023-09-12T14:56:28.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.556+0000 7fe32b39c640 1 --2- 172.21.15.131:0/3355731956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe30c005190 0x7fe30c00b990 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.556+0000 7fe32b39c640 1 -- 172.21.15.131:0/3355731956 >> 172.21.15.131:0/3355731956 conn(0x7fe32406ab10 msgr2=0x7fe3240af730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:28.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.556+0000 7fe32b39c640 1 -- 172.21.15.131:0/3355731956 shutdown_connections 2023-09-12T14:56:28.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.556+0000 7fe32b39c640 1 -- 172.21.15.131:0/3355731956 wait complete. 2023-09-12T14:56:28.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.556+0000 7fe32b39c640 1 Processor -- start 2023-09-12T14:56:28.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.556+0000 7fe32b39c640 1 -- start start 2023-09-12T14:56:28.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.556+0000 7fe32b39c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe30c005190 0x7fe3241f4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:28.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.556+0000 7fe32b39c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe32414bfa0 0x7fe3240c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:28.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.556+0000 7fe32b39c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3241f0f40 0x7fe3240b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:28.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.556+0000 7fe32b39c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32414c740 con 0x7fe32414bfa0 2023-09-12T14:56:28.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.556+0000 7fe32b39c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32414ca20 con 0x7fe3241f0f40 2023-09-12T14:56:28.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.556+0000 7fe32b39c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe32414cd00 con 0x7fe30c005190 2023-09-12T14:56:28.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.556+0000 7fe328937640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe32414bfa0 0x7fe3240c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:28.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.556+0000 7fe328937640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe32414bfa0 0x7fe3240c8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40484/0 (socket says 172.21.15.131:40484) 2023-09-12T14:56:28.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.556+0000 7fe328937640 1 -- 172.21.15.131:0/1698933967 learned_addr learned my addr 172.21.15.131:0/1698933967 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:28.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.560+0000 7fe329939640 1 --2- 172.21.15.131:0/1698933967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3241f0f40 0x7fe3240b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:28.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.560+0000 7fe329138640 1 --2- 172.21.15.131:0/1698933967 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe30c005190 0x7fe3241f4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:28.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.560+0000 7fe328937640 1 -- 172.21.15.131:0/1698933967 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe30c005190 msgr2=0x7fe3241f4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:28.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.560+0000 7fe328937640 1 --2- 172.21.15.131:0/1698933967 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe30c005190 0x7fe3241f4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.560+0000 7fe328937640 1 -- 172.21.15.131:0/1698933967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3241f0f40 msgr2=0x7fe3240b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:28.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.560+0000 7fe328937640 1 --2- 172.21.15.131:0/1698933967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3241f0f40 0x7fe3240b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.560+0000 7fe328937640 1 -- 172.21.15.131:0/1698933967 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3241f0a20 con 0x7fe32414bfa0 2023-09-12T14:56:28.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.560+0000 7fe328937640 1 --2- 172.21.15.131:0/1698933967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe32414bfa0 0x7fe3240c8550 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe31800c400 tx=0x7fe31800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:28.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.560+0000 7fe321ffb640 1 -- 172.21.15.131:0/1698933967 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3180052d0 con 0x7fe32414bfa0 2023-09-12T14:56:28.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.560+0000 7fe321ffb640 1 -- 172.21.15.131:0/1698933967 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe318003600 con 0x7fe32414bfa0 2023-09-12T14:56:28.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.560+0000 7fe32b39c640 1 -- 172.21.15.131:0/1698933967 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe32414c420 con 0x7fe32414bfa0 2023-09-12T14:56:28.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.560+0000 7fe321ffb640 1 -- 172.21.15.131:0/1698933967 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe318003a10 con 0x7fe32414bfa0 2023-09-12T14:56:28.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.560+0000 7fe32b39c640 1 -- 172.21.15.131:0/1698933967 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3240c8c60 con 0x7fe32414bfa0 2023-09-12T14:56:28.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.560+0000 7fe32b39c640 1 -- 172.21.15.131:0/1698933967 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe324073580 con 0x7fe32414bfa0 2023-09-12T14:56:28.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.560+0000 7fe321ffb640 1 -- 172.21.15.131:0/1698933967 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fe318003b70 con 0x7fe32414bfa0 2023-09-12T14:56:28.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.560+0000 7fe321ffb640 1 --2- 172.21.15.131:0/1698933967 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe310065b20 0x7fe310067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:28.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.560+0000 7fe321ffb640 1 -- 172.21.15.131:0/1698933967 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe318082600 con 0x7fe32414bfa0 2023-09-12T14:56:28.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.560+0000 7fe329138640 1 --2- 172.21.15.131:0/1698933967 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe310065b20 0x7fe310067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:28.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.560+0000 7fe329138640 1 --2- 172.21.15.131:0/1698933967 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe310065b20 0x7fe310067fe0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009ab0 tx=0x7fe31c0091e0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:28.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.564+0000 7fe321ffb640 1 -- 172.21.15.131:0/1698933967 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fe3180502e0 con 0x7fe32414bfa0 2023-09-12T14:56:28.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.720+0000 7fe32b39c640 1 -- 172.21.15.131:0/1698933967 --> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fe32414cb90 con 0x7fe310065b20 2023-09-12T14:56:28.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.720+0000 7fe321ffb640 1 -- 172.21.15.131:0/1698933967 <== mgr.4200 v2:172.21.15.131:6832/2298103651 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe32414cb90 con 0x7fe310065b20 2023-09-12T14:56:28.722 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 4 to deep-scrub 2023-09-12T14:56:28.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.720+0000 7fe32b39c640 1 -- 172.21.15.131:0/1698933967 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe310065b20 msgr2=0x7fe310067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:28.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.720+0000 7fe32b39c640 1 --2- 172.21.15.131:0/1698933967 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe310065b20 0x7fe310067fe0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009ab0 tx=0x7fe31c0091e0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.720+0000 7fe32b39c640 1 -- 172.21.15.131:0/1698933967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe32414bfa0 msgr2=0x7fe3240c8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:28.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.720+0000 7fe32b39c640 1 --2- 172.21.15.131:0/1698933967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe32414bfa0 0x7fe3240c8550 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe31800c400 tx=0x7fe31800c8c0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.720+0000 7fe32b39c640 1 -- 172.21.15.131:0/1698933967 shutdown_connections 2023-09-12T14:56:28.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.720+0000 7fe32b39c640 1 --2- 172.21.15.131:0/1698933967 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fe310065b20 0x7fe310067fe0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.720+0000 7fe32b39c640 1 --2- 172.21.15.131:0/1698933967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3241f0f40 0x7fe3240b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.720+0000 7fe32b39c640 1 --2- 172.21.15.131:0/1698933967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe32414bfa0 0x7fe3240c8550 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.720+0000 7fe32b39c640 1 --2- 172.21.15.131:0/1698933967 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe30c005190 0x7fe3241f4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.720+0000 7fe32b39c640 1 -- 172.21.15.131:0/1698933967 >> 172.21.15.131:0/1698933967 conn(0x7fe32406ab10 msgr2=0x7fe32413ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:28.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.720+0000 7fe32b39c640 1 -- 172.21.15.131:0/1698933967 shutdown_connections 2023-09-12T14:56:28.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.720+0000 7fe32b39c640 1 -- 172.21.15.131:0/1698933967 wait complete. 2023-09-12T14:56:28.741 INFO:tasks.ceph:Scrubbing osd.5 2023-09-12T14:56:28.741 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-09-12T14:56:28.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.868+0000 7f1d2073c640 1 -- 172.21.15.131:0/2921695223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d04003980 msgr2=0x7f1d04003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:28.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.868+0000 7f1d2073c640 1 --2- 172.21.15.131:0/2921695223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d04003980 0x7f1d04003da0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400c260 tx=0x7f1d1401cfb0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.868+0000 7f1d1ecd9640 1 -- 172.21.15.131:0/2921695223 reap_dead start 2023-09-12T14:56:28.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.868+0000 7f1d2073c640 1 -- 172.21.15.131:0/2921695223 shutdown_connections 2023-09-12T14:56:28.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.868+0000 7f1d2073c640 1 -- 172.21.15.131:0/2921695223 >> 172.21.15.131:0/2921695223 conn(0x7f1d18069840 msgr2=0x7f1d180cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:28.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.868+0000 7f1d2073c640 1 -- 172.21.15.131:0/2921695223 shutdown_connections 2023-09-12T14:56:28.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.868+0000 7f1d2073c640 1 -- 172.21.15.131:0/2921695223 wait complete. 2023-09-12T14:56:28.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.872+0000 7f1d2073c640 1 Processor -- start 2023-09-12T14:56:28.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.872+0000 7f1d2073c640 1 -- start start 2023-09-12T14:56:28.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.872+0000 7f1d2073c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d04003980 0x7f1d18147460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:28.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.872+0000 7f1d2073c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d1814afd0 0x7f1d18159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:28.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.872+0000 7f1d2073c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d1815a6b0 0x7f1d180b1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:28.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.872+0000 7f1d2073c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d181479a0 con 0x7f1d1814afd0 2023-09-12T14:56:28.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.872+0000 7f1d2073c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1815a400 con 0x7f1d04003980 2023-09-12T14:56:28.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.872+0000 7f1d2073c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1815af00 con 0x7f1d1815a6b0 2023-09-12T14:56:28.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.872+0000 7f1d1dcd7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d1814afd0 0x7f1d18159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:28.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.872+0000 7f1d1dcd7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d1814afd0 0x7f1d18159ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40496/0 (socket says 172.21.15.131:40496) 2023-09-12T14:56:28.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.872+0000 7f1d1dcd7640 1 -- 172.21.15.131:0/2713003264 learned_addr learned my addr 172.21.15.131:0/2713003264 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:28.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.872+0000 7f1d1ecd9640 1 --2- 172.21.15.131:0/2713003264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d1815a6b0 0x7f1d180b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:28.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.872+0000 7f1d1e4d8640 1 --2- 172.21.15.131:0/2713003264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d04003980 0x7f1d18147460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:28.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.872+0000 7f1d1dcd7640 1 -- 172.21.15.131:0/2713003264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d1815a6b0 msgr2=0x7f1d180b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:28.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.872+0000 7f1d1dcd7640 1 --2- 172.21.15.131:0/2713003264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d1815a6b0 0x7f1d180b1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.872+0000 7f1d1dcd7640 1 -- 172.21.15.131:0/2713003264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d04003980 msgr2=0x7f1d18147460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:28.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.872+0000 7f1d1dcd7640 1 --2- 172.21.15.131:0/2713003264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d04003980 0x7f1d18147460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.872+0000 7f1d1dcd7640 1 -- 172.21.15.131:0/2713003264 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1815b1e0 con 0x7f1d1814afd0 2023-09-12T14:56:28.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.872+0000 7f1d1dcd7640 1 --2- 172.21.15.131:0/2713003264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d1814afd0 0x7f1d18159ec0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800e9a0 tx=0x7f1d0800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:28.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.872+0000 7f1d0effd640 1 -- 172.21.15.131:0/2713003264 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0800cc60 con 0x7f1d1814afd0 2023-09-12T14:56:28.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.872+0000 7f1d0effd640 1 -- 172.21.15.131:0/2713003264 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1d0800cdc0 con 0x7f1d1814afd0 2023-09-12T14:56:28.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.872+0000 7f1d2073c640 1 -- 172.21.15.131:0/2713003264 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d181593e0 con 0x7f1d1814afd0 2023-09-12T14:56:28.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.872+0000 7f1d0effd640 1 -- 172.21.15.131:0/2713003264 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d08005280 con 0x7f1d1814afd0 2023-09-12T14:56:28.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.872+0000 7f1d2073c640 1 -- 172.21.15.131:0/2713003264 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d181599a0 con 0x7f1d1814afd0 2023-09-12T14:56:28.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.872+0000 7f1d2073c640 1 -- 172.21.15.131:0/2713003264 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1d18159240 con 0x7f1d1814afd0 2023-09-12T14:56:28.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.876+0000 7f1d0effd640 1 -- 172.21.15.131:0/2713003264 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f1d080053e0 con 0x7f1d1814afd0 2023-09-12T14:56:28.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.876+0000 7f1d0effd640 1 --2- 172.21.15.131:0/2713003264 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1cfc065b50 0x7f1cfc068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:28.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.876+0000 7f1d1e4d8640 1 --2- 172.21.15.131:0/2713003264 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1cfc065b50 0x7f1cfc068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:28.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.876+0000 7f1d0effd640 1 -- 172.21.15.131:0/2713003264 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1d08014070 con 0x7f1d1814afd0 2023-09-12T14:56:28.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.876+0000 7f1d0effd640 1 --2- 172.21.15.131:0/2713003264 >> [v2:172.21.15.187:6824/2170805852,v1:172.21.15.187:6825/2170805852] conn(0x7f1cfc06b9b0 0x7f1cfc06ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:28.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.876+0000 7f1d0effd640 1 -- 172.21.15.131:0/2713003264 --> [v2:172.21.15.187:6824/2170805852,v1:172.21.15.187:6825/2170805852] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1cfc06e480 con 0x7f1cfc06b9b0 2023-09-12T14:56:28.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.876+0000 7f1d0effd640 1 -- 172.21.15.131:0/2713003264 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d08084110 con 0x7f1d1814afd0 2023-09-12T14:56:28.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.876+0000 7f1d1e4d8640 1 --2- 172.21.15.131:0/2713003264 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1cfc065b50 0x7f1cfc068010 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1d140ac000 tx=0x7f1d14019d50 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:28.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.876+0000 7f1d1ecd9640 1 --2- 172.21.15.131:0/2713003264 >> [v2:172.21.15.187:6824/2170805852,v1:172.21.15.187:6825/2170805852] conn(0x7f1cfc06b9b0 0x7f1cfc06ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:28.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.876+0000 7f1d1ecd9640 1 --2- 172.21.15.131:0/2713003264 >> [v2:172.21.15.187:6824/2170805852,v1:172.21.15.187:6825/2170805852] conn(0x7f1cfc06b9b0 0x7f1cfc06ddd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:28.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.876+0000 7f1d0effd640 1 -- 172.21.15.131:0/2713003264 <== osd.5 v2:172.21.15.187:6824/2170805852 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f1cfc06e480 con 0x7f1cfc06b9b0 2023-09-12T14:56:28.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.892+0000 7f1d2073c640 1 -- 172.21.15.131:0/2713003264 --> [v2:172.21.15.187:6824/2170805852,v1:172.21.15.187:6825/2170805852] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1d1815a540 con 0x7f1cfc06b9b0 2023-09-12T14:56:28.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.892+0000 7f1d0effd640 1 -- 172.21.15.131:0/2713003264 <== osd.5 v2:172.21.15.187:6824/2170805852 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f1d1815a540 con 0x7f1cfc06b9b0 2023-09-12T14:56:28.894 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-09-12T14:56:28.895 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-12T14:56:28.895 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-09-12T14:56:28.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.892+0000 7f1d2073c640 1 -- 172.21.15.131:0/2713003264 >> [v2:172.21.15.187:6824/2170805852,v1:172.21.15.187:6825/2170805852] conn(0x7f1cfc06b9b0 msgr2=0x7f1cfc06ddd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:28.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.892+0000 7f1d2073c640 1 --2- 172.21.15.131:0/2713003264 >> [v2:172.21.15.187:6824/2170805852,v1:172.21.15.187:6825/2170805852] conn(0x7f1cfc06b9b0 0x7f1cfc06ddd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:28.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.892+0000 7f1d2073c640 1 -- 172.21.15.131:0/2713003264 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1cfc065b50 msgr2=0x7f1cfc068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:28.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.892+0000 7f1d2073c640 1 --2- 172.21.15.131:0/2713003264 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f1cfc065b50 0x7f1cfc068010 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1d140ac000 tx=0x7f1d14019d50 comp rx=0 tx=0).stop 2023-09-12T14:56:28.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.892+0000 7f1d2073c640 1 -- 172.21.15.131:0/2713003264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d1814afd0 msgr2=0x7f1d18159ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:28.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.892+0000 7f1d2073c640 1 --2- 172.21.15.131:0/2713003264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d1814afd0 0x7f1d18159ec0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800e9a0 tx=0x7f1d0800ee60 comp rx=0 tx=0).stop 2023-09-12T14:56:28.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.892+0000 7f1d1ecd9640 1 -- 172.21.15.131:0/2713003264 reap_dead start 2023-09-12T14:56:28.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.892+0000 7f1d2073c640 1 -- 172.21.15.131:0/2713003264 shutdown_connections 2023-09-12T14:56:28.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.892+0000 7f1d2073c640 1 -- 172.21.15.131:0/2713003264 >> 172.21.15.131:0/2713003264 conn(0x7f1d18069840 msgr2=0x7f1d1806f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:28.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.892+0000 7f1d2073c640 1 -- 172.21.15.131:0/2713003264 shutdown_connections 2023-09-12T14:56:28.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:28.892+0000 7f1d2073c640 1 -- 172.21.15.131:0/2713003264 wait complete. 2023-09-12T14:56:28.910 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-09-12T14:56:29.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.028+0000 7f71b8300640 1 -- 172.21.15.131:0/1550010339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f719c003980 msgr2=0x7f719c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:29.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.028+0000 7f71b8300640 1 --2- 172.21.15.131:0/1550010339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f719c003980 0x7f719c003da0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00bb40 tx=0x7f71ac01cd50 comp rx=0 tx=0).stop 2023-09-12T14:56:29.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.028+0000 7f71b689d640 1 -- 172.21.15.131:0/1550010339 reap_dead start 2023-09-12T14:56:29.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.028+0000 7f71b8300640 1 -- 172.21.15.131:0/1550010339 shutdown_connections 2023-09-12T14:56:29.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.028+0000 7f71b8300640 1 -- 172.21.15.131:0/1550010339 >> 172.21.15.131:0/1550010339 conn(0x7f71b0069840 msgr2=0x7f71b00cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:29.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.028+0000 7f71b8300640 1 -- 172.21.15.131:0/1550010339 shutdown_connections 2023-09-12T14:56:29.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.028+0000 7f71b8300640 1 -- 172.21.15.131:0/1550010339 wait complete. 2023-09-12T14:56:29.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.028+0000 7f71b8300640 1 Processor -- start 2023-09-12T14:56:29.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.028+0000 7f71b8300640 1 -- start start 2023-09-12T14:56:29.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71b8300640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f719c003980 0x7f71b01413c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:29.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71b8300640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71b013f330 0x7f71b00cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:29.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71b8300640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71b013ea20 0x7f71b0150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:29.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71b8300640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b00cc090 con 0x7f71b013f330 2023-09-12T14:56:29.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71b8300640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b00ae3b0 con 0x7f71b013ea20 2023-09-12T14:56:29.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71b8300640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b0142320 con 0x7f719c003980 2023-09-12T14:56:29.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71b589b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71b013f330 0x7f71b00cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71b589b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71b013f330 0x7f71b00cbb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40504/0 (socket says 172.21.15.131:40504) 2023-09-12T14:56:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71b589b640 1 -- 172.21.15.131:0/2268374898 learned_addr learned my addr 172.21.15.131:0/2268374898 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71b609c640 1 --2- 172.21.15.131:0/2268374898 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f719c003980 0x7f71b01413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71b689d640 1 --2- 172.21.15.131:0/2268374898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71b013ea20 0x7f71b0150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:29.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71b589b640 1 -- 172.21.15.131:0/2268374898 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f719c003980 msgr2=0x7f71b01413c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:29.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71b589b640 1 --2- 172.21.15.131:0/2268374898 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f719c003980 0x7f71b01413c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71b589b640 1 -- 172.21.15.131:0/2268374898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71b013ea20 msgr2=0x7f71b0150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:29.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71b589b640 1 --2- 172.21.15.131:0/2268374898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71b013ea20 0x7f71b0150f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71b589b640 1 -- 172.21.15.131:0/2268374898 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b013f7b0 con 0x7f71b013f330 2023-09-12T14:56:29.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71b689d640 1 --2- 172.21.15.131:0/2268374898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71b013ea20 0x7f71b0150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:29.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71b589b640 1 --2- 172.21.15.131:0/2268374898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71b013f330 0x7f71b00cbb50 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f71a000d820 tx=0x7f71a000dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:29.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71a6ffd640 1 -- 172.21.15.131:0/2268374898 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a00052e0 con 0x7f71b013f330 2023-09-12T14:56:29.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71a6ffd640 1 -- 172.21.15.131:0/2268374898 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f71a0005440 con 0x7f71b013f330 2023-09-12T14:56:29.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71b8300640 1 -- 172.21.15.131:0/2268374898 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b0141a80 con 0x7f71b013f330 2023-09-12T14:56:29.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71a6ffd640 1 -- 172.21.15.131:0/2268374898 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a000b8c0 con 0x7f71b013f330 2023-09-12T14:56:29.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71b8300640 1 -- 172.21.15.131:0/2268374898 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b0142040 con 0x7f71b013f330 2023-09-12T14:56:29.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71b8300640 1 -- 172.21.15.131:0/2268374898 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b0073580 con 0x7f71b013f330 2023-09-12T14:56:29.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71a6ffd640 1 -- 172.21.15.131:0/2268374898 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f71a000ba20 con 0x7f71b013f330 2023-09-12T14:56:29.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71a6ffd640 1 --2- 172.21.15.131:0/2268374898 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7194065b20 0x7f7194067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:29.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71a6ffd640 1 -- 172.21.15.131:0/2268374898 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f71a00829e0 con 0x7f71b013f330 2023-09-12T14:56:29.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.032+0000 7f71b609c640 1 --2- 172.21.15.131:0/2268374898 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7194065b20 0x7f7194067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:29.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.036+0000 7f71b609c640 1 --2- 172.21.15.131:0/2268374898 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7194065b20 0x7f7194067fe0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0b0000 tx=0x7f71ac0a7040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:29.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.036+0000 7f71a6ffd640 1 -- 172.21.15.131:0/2268374898 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f71a0050630 con 0x7f71b013f330 2023-09-12T14:56:29.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.180+0000 7f71b8300640 1 -- 172.21.15.131:0/2268374898 --> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f71b00cd4d0 con 0x7f7194065b20 2023-09-12T14:56:29.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.180+0000 7f71a6ffd640 1 -- 172.21.15.131:0/2268374898 <== mgr.4200 v2:172.21.15.131:6832/2298103651 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f71b00cd4d0 con 0x7f7194065b20 2023-09-12T14:56:29.185 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 5 to deep-scrub 2023-09-12T14:56:29.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.184+0000 7f71b8300640 1 -- 172.21.15.131:0/2268374898 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7194065b20 msgr2=0x7f7194067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:29.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.184+0000 7f71b8300640 1 --2- 172.21.15.131:0/2268374898 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7194065b20 0x7f7194067fe0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0b0000 tx=0x7f71ac0a7040 comp rx=0 tx=0).stop 2023-09-12T14:56:29.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.184+0000 7f71b8300640 1 -- 172.21.15.131:0/2268374898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71b013f330 msgr2=0x7f71b00cbb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:29.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.184+0000 7f71b8300640 1 --2- 172.21.15.131:0/2268374898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71b013f330 0x7f71b00cbb50 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f71a000d820 tx=0x7f71a000dce0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.184+0000 7f71b8300640 1 -- 172.21.15.131:0/2268374898 shutdown_connections 2023-09-12T14:56:29.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.184+0000 7f71b8300640 1 --2- 172.21.15.131:0/2268374898 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f7194065b20 0x7f7194067fe0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.184+0000 7f71b8300640 1 --2- 172.21.15.131:0/2268374898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71b013ea20 0x7f71b0150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.184+0000 7f71b8300640 1 --2- 172.21.15.131:0/2268374898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71b013f330 0x7f71b00cbb50 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.184+0000 7f71b8300640 1 --2- 172.21.15.131:0/2268374898 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f719c003980 0x7f71b01413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.189 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.184+0000 7f71b8300640 1 -- 172.21.15.131:0/2268374898 >> 172.21.15.131:0/2268374898 conn(0x7f71b0069840 msgr2=0x7f71b01584c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:29.189 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.184+0000 7f71b8300640 1 -- 172.21.15.131:0/2268374898 shutdown_connections 2023-09-12T14:56:29.189 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.184+0000 7f71b8300640 1 -- 172.21.15.131:0/2268374898 wait complete. 2023-09-12T14:56:29.201 INFO:tasks.ceph:Scrubbing osd.6 2023-09-12T14:56:29.202 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-09-12T14:56:29.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.328+0000 7f47684b7640 1 -- 172.21.15.131:0/2397501046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47600c7700 msgr2=0x7f47600c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:29.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.328+0000 7f47684b7640 1 --2- 172.21.15.131:0/2397501046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47600c7700 0x7f47600c7b90 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4760071770 tx=0x7f475001c700 comp rx=0 tx=0).stop 2023-09-12T14:56:29.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.328+0000 7f47684b7640 1 -- 172.21.15.131:0/2397501046 shutdown_connections 2023-09-12T14:56:29.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.328+0000 7f47684b7640 1 --2- 172.21.15.131:0/2397501046 >> v2:172.21.15.187:3300/0 conn(0x7f4760069240 0x7f4760069660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.328+0000 7f47684b7640 1 --2- 172.21.15.131:0/2397501046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47600c7700 0x7f47600c7b90 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.328+0000 7f47684b7640 1 -- 172.21.15.131:0/2397501046 >> 172.21.15.131:0/2397501046 conn(0x7f476006b310 msgr2=0x7f476006b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:29.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.328+0000 7f47684b7640 1 -- 172.21.15.131:0/2397501046 shutdown_connections 2023-09-12T14:56:29.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.328+0000 7f47684b7640 1 -- 172.21.15.131:0/2397501046 wait complete. 2023-09-12T14:56:29.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.332+0000 7f47684b7640 1 Processor -- start 2023-09-12T14:56:29.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.332+0000 7f47684b7640 1 -- start start 2023-09-12T14:56:29.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.332+0000 7f47684b7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4760069240 0x7f47600b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:29.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.332+0000 7f47684b7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47600c7700 0x7f476014d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:29.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.332+0000 7f47684b7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f476014e220 0x7f476014d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:29.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.332+0000 7f47684b7640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47600b1db0 con 0x7f476014e220 2023-09-12T14:56:29.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.332+0000 7f47684b7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47600b23f0 con 0x7f47600c7700 2023-09-12T14:56:29.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.332+0000 7f47684b7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47600b2640 con 0x7f4760069240 2023-09-12T14:56:29.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.332+0000 7f4766a54640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f476014e220 0x7f476014d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:29.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.332+0000 7f4766a54640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f476014e220 0x7f476014d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40518/0 (socket says 172.21.15.131:40518) 2023-09-12T14:56:29.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.332+0000 7f4766a54640 1 -- 172.21.15.131:0/4249220485 learned_addr learned my addr 172.21.15.131:0/4249220485 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:29.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.332+0000 7f4765a52640 1 --2- 172.21.15.131:0/4249220485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47600c7700 0x7f476014d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:29.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.332+0000 7f4766253640 1 --2- 172.21.15.131:0/4249220485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4760069240 0x7f47600b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:29.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.332+0000 7f4766a54640 1 -- 172.21.15.131:0/4249220485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4760069240 msgr2=0x7f47600b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:29.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.332+0000 7f4766a54640 1 --2- 172.21.15.131:0/4249220485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4760069240 0x7f47600b1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.332+0000 7f4766a54640 1 -- 172.21.15.131:0/4249220485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47600c7700 msgr2=0x7f476014d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:29.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.332+0000 7f4766a54640 1 --2- 172.21.15.131:0/4249220485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47600c7700 0x7f476014d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.332+0000 7f4766a54640 1 -- 172.21.15.131:0/4249220485 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47500094a0 con 0x7f476014e220 2023-09-12T14:56:29.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.332+0000 7f4766253640 1 --2- 172.21.15.131:0/4249220485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4760069240 0x7f47600b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:29.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.332+0000 7f4766a54640 1 --2- 172.21.15.131:0/4249220485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f476014e220 0x7f476014d6f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f475800ea20 tx=0x7f475800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:29.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.332+0000 7f4756ffd640 1 -- 172.21.15.131:0/4249220485 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475800cde0 con 0x7f476014e220 2023-09-12T14:56:29.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.332+0000 7f4756ffd640 1 -- 172.21.15.131:0/4249220485 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4758004c40 con 0x7f476014e220 2023-09-12T14:56:29.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.332+0000 7f47684b7640 1 -- 172.21.15.131:0/4249220485 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47600b2920 con 0x7f476014e220 2023-09-12T14:56:29.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.332+0000 7f4756ffd640 1 -- 172.21.15.131:0/4249220485 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4758005020 con 0x7f476014e220 2023-09-12T14:56:29.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.332+0000 7f47684b7640 1 -- 172.21.15.131:0/4249220485 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47600b1350 con 0x7f476014e220 2023-09-12T14:56:29.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.332+0000 7f47684b7640 1 -- 172.21.15.131:0/4249220485 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4760069fb0 con 0x7f476014e220 2023-09-12T14:56:29.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.336+0000 7f4756ffd640 1 -- 172.21.15.131:0/4249220485 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f4758005180 con 0x7f476014e220 2023-09-12T14:56:29.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.336+0000 7f4756ffd640 1 --2- 172.21.15.131:0/4249220485 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4744065b50 0x7f4744068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:29.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.336+0000 7f4756ffd640 1 -- 172.21.15.131:0/4249220485 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4758083a10 con 0x7f476014e220 2023-09-12T14:56:29.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.336+0000 7f4766253640 1 --2- 172.21.15.131:0/4249220485 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4744065b50 0x7f4744068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:29.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.336+0000 7f4756ffd640 1 --2- 172.21.15.131:0/4249220485 >> [v2:172.21.15.187:6801/221066879,v1:172.21.15.187:6803/221066879] conn(0x7f474406b9b0 0x7f474406ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:29.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.336+0000 7f4756ffd640 1 -- 172.21.15.131:0/4249220485 --> [v2:172.21.15.187:6801/221066879,v1:172.21.15.187:6803/221066879] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f474406e480 con 0x7f474406b9b0 2023-09-12T14:56:29.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.336+0000 7f4756ffd640 1 -- 172.21.15.131:0/4249220485 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47580882c0 con 0x7f476014e220 2023-09-12T14:56:29.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.336+0000 7f4766253640 1 --2- 172.21.15.131:0/4249220485 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4744065b50 0x7f4744068010 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f475c0011f0 tx=0x7f475c009040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:29.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.336+0000 7f4765a52640 1 --2- 172.21.15.131:0/4249220485 >> [v2:172.21.15.187:6801/221066879,v1:172.21.15.187:6803/221066879] conn(0x7f474406b9b0 0x7f474406ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:29.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.336+0000 7f4765a52640 1 --2- 172.21.15.131:0/4249220485 >> [v2:172.21.15.187:6801/221066879,v1:172.21.15.187:6803/221066879] conn(0x7f474406b9b0 0x7f474406ddd0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:29.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.336+0000 7f4756ffd640 1 -- 172.21.15.131:0/4249220485 <== osd.6 v2:172.21.15.187:6801/221066879 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f474406e480 con 0x7f474406b9b0 2023-09-12T14:56:29.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.352+0000 7f47684b7640 1 -- 172.21.15.131:0/4249220485 --> [v2:172.21.15.187:6801/221066879,v1:172.21.15.187:6803/221066879] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f47600b2b60 con 0x7f474406b9b0 2023-09-12T14:56:29.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.352+0000 7f4756ffd640 1 -- 172.21.15.131:0/4249220485 <== osd.6 v2:172.21.15.187:6801/221066879 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f47600b2b60 con 0x7f474406b9b0 2023-09-12T14:56:29.374 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-09-12T14:56:29.374 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-12T14:56:29.374 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-09-12T14:56:29.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.352+0000 7f47684b7640 1 -- 172.21.15.131:0/4249220485 >> [v2:172.21.15.187:6801/221066879,v1:172.21.15.187:6803/221066879] conn(0x7f474406b9b0 msgr2=0x7f474406ddd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:29.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.352+0000 7f47684b7640 1 --2- 172.21.15.131:0/4249220485 >> [v2:172.21.15.187:6801/221066879,v1:172.21.15.187:6803/221066879] conn(0x7f474406b9b0 0x7f474406ddd0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.352+0000 7f47684b7640 1 -- 172.21.15.131:0/4249220485 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4744065b50 msgr2=0x7f4744068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:29.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.352+0000 7f47684b7640 1 --2- 172.21.15.131:0/4249220485 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f4744065b50 0x7f4744068010 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f475c0011f0 tx=0x7f475c009040 comp rx=0 tx=0).stop 2023-09-12T14:56:29.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.352+0000 7f47684b7640 1 -- 172.21.15.131:0/4249220485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f476014e220 msgr2=0x7f476014d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:29.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.352+0000 7f47684b7640 1 --2- 172.21.15.131:0/4249220485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f476014e220 0x7f476014d6f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f475800ea20 tx=0x7f475800eee0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.352+0000 7f4766a54640 1 -- 172.21.15.131:0/4249220485 reap_dead start 2023-09-12T14:56:29.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.352+0000 7f47684b7640 1 -- 172.21.15.131:0/4249220485 shutdown_connections 2023-09-12T14:56:29.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.352+0000 7f47684b7640 1 -- 172.21.15.131:0/4249220485 >> 172.21.15.131:0/4249220485 conn(0x7f476006b310 msgr2=0x7f4760158e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:29.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.352+0000 7f47684b7640 1 -- 172.21.15.131:0/4249220485 shutdown_connections 2023-09-12T14:56:29.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.352+0000 7f47684b7640 1 -- 172.21.15.131:0/4249220485 wait complete. 2023-09-12T14:56:29.379 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-09-12T14:56:29.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.496+0000 7f72f1f32640 1 -- 172.21.15.131:0/322377498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d80052b0 msgr2=0x7f72d800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:29.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.496+0000 7f72f1f32640 1 --2- 172.21.15.131:0/322377498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d80052b0 0x7f72d800b990 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f72e4005fd0 tx=0x7f72e401cda0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.496+0000 7f72f1f32640 1 -- 172.21.15.131:0/322377498 shutdown_connections 2023-09-12T14:56:29.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.496+0000 7f72f1f32640 1 --2- 172.21.15.131:0/322377498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d80052b0 0x7f72d800b990 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.496+0000 7f72f1f32640 1 -- 172.21.15.131:0/322377498 >> 172.21.15.131:0/322377498 conn(0x7f72ec069840 msgr2=0x7f72ec0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:29.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.496+0000 7f72f1f32640 1 -- 172.21.15.131:0/322377498 shutdown_connections 2023-09-12T14:56:29.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.496+0000 7f72f1f32640 1 -- 172.21.15.131:0/322377498 wait complete. 2023-09-12T14:56:29.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.496+0000 7f72f1f32640 1 Processor -- start 2023-09-12T14:56:29.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.496+0000 7f72f1f32640 1 -- start start 2023-09-12T14:56:29.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72f1f32640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d80052b0 0x7f72ec14f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:29.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72f1f32640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72ec1536c0 0x7f72ec1e4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:29.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72f1f32640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72ec1e4ad0 0x7f72ec153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:29.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72f1f32640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec1e4f80 con 0x7f72d80052b0 2023-09-12T14:56:29.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72f1f32640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec152620 con 0x7f72ec1e4ad0 2023-09-12T14:56:29.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72f1f32640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec152900 con 0x7f72ec1536c0 2023-09-12T14:56:29.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72eb7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d80052b0 0x7f72ec14f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:29.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72eb7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d80052b0 0x7f72ec14f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40540/0 (socket says 172.21.15.131:40540) 2023-09-12T14:56:29.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72eb7fe640 1 -- 172.21.15.131:0/3626381468 learned_addr learned my addr 172.21.15.131:0/3626381468 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:29.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72ebfff640 1 --2- 172.21.15.131:0/3626381468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72ec1e4ad0 0x7f72ec153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:29.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72eaffd640 1 --2- 172.21.15.131:0/3626381468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72ec1536c0 0x7f72ec1e4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:29.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72eb7fe640 1 -- 172.21.15.131:0/3626381468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72ec1536c0 msgr2=0x7f72ec1e4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:29.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72eb7fe640 1 --2- 172.21.15.131:0/3626381468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72ec1536c0 0x7f72ec1e4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72eb7fe640 1 -- 172.21.15.131:0/3626381468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72ec1e4ad0 msgr2=0x7f72ec153b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:29.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72eb7fe640 1 --2- 172.21.15.131:0/3626381468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72ec1e4ad0 0x7f72ec153b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72eb7fe640 1 -- 172.21.15.131:0/3626381468 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ec152be0 con 0x7f72d80052b0 2023-09-12T14:56:29.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72ebfff640 1 --2- 172.21.15.131:0/3626381468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72ec1e4ad0 0x7f72ec153b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-12T14:56:29.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72eb7fe640 1 --2- 172.21.15.131:0/3626381468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d80052b0 0x7f72ec14f370 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00d870 tx=0x7f72dc00dd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:29.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72e3fff640 1 -- 172.21.15.131:0/3626381468 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc00b8d0 con 0x7f72d80052b0 2023-09-12T14:56:29.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72e3fff640 1 -- 172.21.15.131:0/3626381468 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f72dc00ba50 con 0x7f72d80052b0 2023-09-12T14:56:29.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72e3fff640 1 -- 172.21.15.131:0/3626381468 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc00be60 con 0x7f72d80052b0 2023-09-12T14:56:29.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72f1f32640 1 -- 172.21.15.131:0/3626381468 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72ec152ec0 con 0x7f72d80052b0 2023-09-12T14:56:29.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72f1f32640 1 -- 172.21.15.131:0/3626381468 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72ec14f8b0 con 0x7f72d80052b0 2023-09-12T14:56:29.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72f1f32640 1 -- 172.21.15.131:0/3626381468 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72ec1533e0 con 0x7f72d80052b0 2023-09-12T14:56:29.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72e3fff640 1 -- 172.21.15.131:0/3626381468 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f72dc01b400 con 0x7f72d80052b0 2023-09-12T14:56:29.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72e3fff640 1 --2- 172.21.15.131:0/3626381468 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f72cc065b50 0x7f72cc068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:29.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72e3fff640 1 -- 172.21.15.131:0/3626381468 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f72dc0835d0 con 0x7f72d80052b0 2023-09-12T14:56:29.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72eaffd640 1 --2- 172.21.15.131:0/3626381468 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f72cc065b50 0x7f72cc068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:29.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.500+0000 7f72eaffd640 1 --2- 172.21.15.131:0/3626381468 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f72cc065b50 0x7f72cc068010 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f72ec1e4520 tx=0x7f72d40014f0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:29.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.504+0000 7f72e3fff640 1 -- 172.21.15.131:0/3626381468 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f72ec1533e0 con 0x7f72d80052b0 2023-09-12T14:56:29.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.660+0000 7f72f1f32640 1 -- 172.21.15.131:0/3626381468 --> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f72ec1524a0 con 0x7f72cc065b50 2023-09-12T14:56:29.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.660+0000 7f72e3fff640 1 -- 172.21.15.131:0/3626381468 <== mgr.4200 v2:172.21.15.131:6832/2298103651 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f72d40119c0 con 0x7f72cc065b50 2023-09-12T14:56:29.664 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 6 to deep-scrub 2023-09-12T14:56:29.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.664+0000 7f72f1f32640 1 -- 172.21.15.131:0/3626381468 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f72cc065b50 msgr2=0x7f72cc068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:29.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.664+0000 7f72f1f32640 1 --2- 172.21.15.131:0/3626381468 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f72cc065b50 0x7f72cc068010 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f72ec1e4520 tx=0x7f72d40014f0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.664+0000 7f72f1f32640 1 -- 172.21.15.131:0/3626381468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d80052b0 msgr2=0x7f72ec14f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:29.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.664+0000 7f72f1f32640 1 --2- 172.21.15.131:0/3626381468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d80052b0 0x7f72ec14f370 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00d870 tx=0x7f72dc00dd30 comp rx=0 tx=0).stop 2023-09-12T14:56:29.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.664+0000 7f72f1f32640 1 -- 172.21.15.131:0/3626381468 shutdown_connections 2023-09-12T14:56:29.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.664+0000 7f72f1f32640 1 --2- 172.21.15.131:0/3626381468 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f72cc065b50 0x7f72cc068010 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.664+0000 7f72f1f32640 1 --2- 172.21.15.131:0/3626381468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72ec1536c0 0x7f72ec1e4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.664+0000 7f72f1f32640 1 --2- 172.21.15.131:0/3626381468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72ec1e4ad0 0x7f72ec153b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.664+0000 7f72f1f32640 1 --2- 172.21.15.131:0/3626381468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d80052b0 0x7f72ec14f370 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.664+0000 7f72f1f32640 1 -- 172.21.15.131:0/3626381468 >> 172.21.15.131:0/3626381468 conn(0x7f72ec069840 msgr2=0x7f72ec06fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:29.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.664+0000 7f72f1f32640 1 -- 172.21.15.131:0/3626381468 shutdown_connections 2023-09-12T14:56:29.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.664+0000 7f72f1f32640 1 -- 172.21.15.131:0/3626381468 wait complete. 2023-09-12T14:56:29.682 INFO:tasks.ceph:Scrubbing osd.7 2023-09-12T14:56:29.683 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-09-12T14:56:29.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.804+0000 7f518d5de640 1 -- 172.21.15.131:0/3510150358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51880abae0 msgr2=0x7f518806b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:29.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.804+0000 7f518d5de640 1 --2- 172.21.15.131:0/3510150358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51880abae0 0x7f518806b430 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5180009bc0 tx=0x7f518001cab0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.804+0000 7f518d5de640 1 -- 172.21.15.131:0/3510150358 shutdown_connections 2023-09-12T14:56:29.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.804+0000 7f518d5de640 1 --2- 172.21.15.131:0/3510150358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51880abae0 0x7f518806b430 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.804+0000 7f518d5de640 1 -- 172.21.15.131:0/3510150358 >> 172.21.15.131:0/3510150358 conn(0x7f5188069840 msgr2=0x7f51880cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:29.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.804+0000 7f518d5de640 1 -- 172.21.15.131:0/3510150358 shutdown_connections 2023-09-12T14:56:29.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.804+0000 7f518d5de640 1 -- 172.21.15.131:0/3510150358 wait complete. 2023-09-12T14:56:29.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.804+0000 7f518d5de640 1 Processor -- start 2023-09-12T14:56:29.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.804+0000 7f518d5de640 1 -- start start 2023-09-12T14:56:29.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f518d5de640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51880abae0 0x7f518814a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:29.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f518d5de640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51880c9a20 0x7f518814b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:29.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f518d5de640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5188158d70 0x7f51880aae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:29.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f518d5de640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518814a6a0 con 0x7f51880c9a20 2023-09-12T14:56:29.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f518d5de640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518814b160 con 0x7f51880abae0 2023-09-12T14:56:29.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f518d5de640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5188156480 con 0x7f5188158d70 2023-09-12T14:56:29.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f51867fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51880c9a20 0x7f518814b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:29.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f51867fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51880c9a20 0x7f518814b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40560/0 (socket says 172.21.15.131:40560) 2023-09-12T14:56:29.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f51867fc640 1 -- 172.21.15.131:0/353816509 learned_addr learned my addr 172.21.15.131:0/353816509 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:29.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f51877fe640 1 --2- 172.21.15.131:0/353816509 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5188158d70 0x7f51880aae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:29.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f51867fc640 1 -- 172.21.15.131:0/353816509 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5188158d70 msgr2=0x7f51880aae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:29.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f5186ffd640 1 --2- 172.21.15.131:0/353816509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51880abae0 0x7f518814a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:29.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f51867fc640 1 --2- 172.21.15.131:0/353816509 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5188158d70 0x7f51880aae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f51867fc640 1 -- 172.21.15.131:0/353816509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51880abae0 msgr2=0x7f518814a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:29.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f51867fc640 1 --2- 172.21.15.131:0/353816509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51880abae0 0x7f518814a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f51867fc640 1 -- 172.21.15.131:0/353816509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5180009820 con 0x7f51880c9a20 2023-09-12T14:56:29.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f51867fc640 1 --2- 172.21.15.131:0/353816509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51880c9a20 0x7f518814b770 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f517c00c450 tx=0x7f517c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:29.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f516f7fe640 1 -- 172.21.15.131:0/353816509 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c004230 con 0x7f51880c9a20 2023-09-12T14:56:29.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f516f7fe640 1 -- 172.21.15.131:0/353816509 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f517c0044a0 con 0x7f51880c9a20 2023-09-12T14:56:29.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f518d5de640 1 -- 172.21.15.131:0/353816509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5188157020 con 0x7f51880c9a20 2023-09-12T14:56:29.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f516f7fe640 1 -- 172.21.15.131:0/353816509 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c005260 con 0x7f51880c9a20 2023-09-12T14:56:29.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f518d5de640 1 -- 172.21.15.131:0/353816509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51881575e0 con 0x7f51880c9a20 2023-09-12T14:56:29.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f518d5de640 1 -- 172.21.15.131:0/353816509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f51881675f0 con 0x7f51880c9a20 2023-09-12T14:56:29.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f516f7fe640 1 -- 172.21.15.131:0/353816509 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f517c0053c0 con 0x7f51880c9a20 2023-09-12T14:56:29.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f516f7fe640 1 --2- 172.21.15.131:0/353816509 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5174065b20 0x7f5174067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:29.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f516f7fe640 1 -- 172.21.15.131:0/353816509 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f517c083990 con 0x7f51880c9a20 2023-09-12T14:56:29.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f5186ffd640 1 --2- 172.21.15.131:0/353816509 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5174065b20 0x7f5174067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:29.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f516f7fe640 1 --2- 172.21.15.131:0/353816509 >> [v2:172.21.15.187:6816/827490193,v1:172.21.15.187:6817/827490193] conn(0x7f517406b980 0x7f517406dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:29.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f516f7fe640 1 -- 172.21.15.131:0/353816509 --> [v2:172.21.15.187:6816/827490193,v1:172.21.15.187:6817/827490193] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f517406e450 con 0x7f517406b980 2023-09-12T14:56:29.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f516f7fe640 1 -- 172.21.15.131:0/353816509 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f517c083de0 con 0x7f51880c9a20 2023-09-12T14:56:29.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f5186ffd640 1 --2- 172.21.15.131:0/353816509 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5174065b20 0x7f5174067fe0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f517800a750 tx=0x7f5178008040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:29.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.808+0000 7f51877fe640 1 --2- 172.21.15.131:0/353816509 >> [v2:172.21.15.187:6816/827490193,v1:172.21.15.187:6817/827490193] conn(0x7f517406b980 0x7f517406dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:29.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.812+0000 7f51877fe640 1 --2- 172.21.15.131:0/353816509 >> [v2:172.21.15.187:6816/827490193,v1:172.21.15.187:6817/827490193] conn(0x7f517406b980 0x7f517406dda0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:29.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.812+0000 7f516f7fe640 1 -- 172.21.15.131:0/353816509 <== osd.7 v2:172.21.15.187:6816/827490193 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f517406e450 con 0x7f517406b980 2023-09-12T14:56:29.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.824+0000 7f518d5de640 1 -- 172.21.15.131:0/353816509 --> [v2:172.21.15.187:6816/827490193,v1:172.21.15.187:6817/827490193] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5188073580 con 0x7f517406b980 2023-09-12T14:56:29.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.824+0000 7f516f7fe640 1 -- 172.21.15.131:0/353816509 <== osd.7 v2:172.21.15.187:6816/827490193 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f5188073580 con 0x7f517406b980 2023-09-12T14:56:29.829 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-09-12T14:56:29.829 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-12T14:56:29.829 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-09-12T14:56:29.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.824+0000 7f518d5de640 1 -- 172.21.15.131:0/353816509 >> [v2:172.21.15.187:6816/827490193,v1:172.21.15.187:6817/827490193] conn(0x7f517406b980 msgr2=0x7f517406dda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:29.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.824+0000 7f518d5de640 1 --2- 172.21.15.131:0/353816509 >> [v2:172.21.15.187:6816/827490193,v1:172.21.15.187:6817/827490193] conn(0x7f517406b980 0x7f517406dda0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.828+0000 7f518d5de640 1 -- 172.21.15.131:0/353816509 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5174065b20 msgr2=0x7f5174067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:29.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.828+0000 7f518d5de640 1 --2- 172.21.15.131:0/353816509 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5174065b20 0x7f5174067fe0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f517800a750 tx=0x7f5178008040 comp rx=0 tx=0).stop 2023-09-12T14:56:29.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.828+0000 7f518d5de640 1 -- 172.21.15.131:0/353816509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51880c9a20 msgr2=0x7f518814b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:29.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.828+0000 7f518d5de640 1 --2- 172.21.15.131:0/353816509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51880c9a20 0x7f518814b770 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f517c00c450 tx=0x7f517c00c910 comp rx=0 tx=0).stop 2023-09-12T14:56:29.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.828+0000 7f51877fe640 1 -- 172.21.15.131:0/353816509 reap_dead start 2023-09-12T14:56:29.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.828+0000 7f518d5de640 1 -- 172.21.15.131:0/353816509 shutdown_connections 2023-09-12T14:56:29.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.828+0000 7f518d5de640 1 -- 172.21.15.131:0/353816509 >> 172.21.15.131:0/353816509 conn(0x7f5188069840 msgr2=0x7f51880ac620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:29.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.828+0000 7f518d5de640 1 -- 172.21.15.131:0/353816509 shutdown_connections 2023-09-12T14:56:29.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.828+0000 7f518d5de640 1 -- 172.21.15.131:0/353816509 wait complete. 2023-09-12T14:56:29.844 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-09-12T14:56:29.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.948+0000 7f5fab6be640 1 -- 172.21.15.131:0/1053582656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fa40c9370 msgr2=0x7f5fa40c9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:29.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.948+0000 7f5fab6be640 1 --2- 172.21.15.131:0/1053582656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fa40c9370 0x7f5fa40c9780 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40cd200 tx=0x7f5f9c01ccf0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.948+0000 7f5fab6be640 1 -- 172.21.15.131:0/1053582656 shutdown_connections 2023-09-12T14:56:29.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.948+0000 7f5fab6be640 1 --2- 172.21.15.131:0/1053582656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fa40c9370 0x7f5fa40c9780 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.948+0000 7f5fab6be640 1 --2- 172.21.15.131:0/1053582656 >> v2:172.21.15.187:3300/0 conn(0x7f5fa4156930 0x7f5fa4159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.948+0000 7f5fab6be640 1 -- 172.21.15.131:0/1053582656 >> 172.21.15.131:0/1053582656 conn(0x7f5fa4069840 msgr2=0x7f5fa40cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:29.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.948+0000 7f5fab6be640 1 -- 172.21.15.131:0/1053582656 shutdown_connections 2023-09-12T14:56:29.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.948+0000 7f5fab6be640 1 -- 172.21.15.131:0/1053582656 wait complete. 2023-09-12T14:56:29.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.952+0000 7f5fab6be640 1 Processor -- start 2023-09-12T14:56:29.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.952+0000 7f5fab6be640 1 -- start start 2023-09-12T14:56:29.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.952+0000 7f5fab6be640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5fa40abb40 0x7f5fa41485a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:29.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.952+0000 7f5fab6be640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fa40c9370 0x7f5fa414bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:29.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.952+0000 7f5fab6be640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5fa4156930 0x7f5fa414c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:29.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.952+0000 7f5fab6be640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa4151080 con 0x7f5fa40c9370 2023-09-12T14:56:29.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.952+0000 7f5fab6be640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa406ef90 con 0x7f5fa4156930 2023-09-12T14:56:29.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.952+0000 7f5fab6be640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa4148ae0 con 0x7f5fa40abb40 2023-09-12T14:56:29.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.952+0000 7f5fa8c59640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fa40c9370 0x7f5fa414bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:29.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.952+0000 7f5fa8c59640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fa40c9370 0x7f5fa414bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40582/0 (socket says 172.21.15.131:40582) 2023-09-12T14:56:29.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.952+0000 7f5fa8c59640 1 -- 172.21.15.131:0/4202070596 learned_addr learned my addr 172.21.15.131:0/4202070596 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:29.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.952+0000 7f5fa945a640 1 --2- 172.21.15.131:0/4202070596 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5fa40abb40 0x7f5fa41485a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:29.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.952+0000 7f5fa8c59640 1 -- 172.21.15.131:0/4202070596 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5fa40abb40 msgr2=0x7f5fa41485a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:29.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.952+0000 7f5fa8c59640 1 --2- 172.21.15.131:0/4202070596 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5fa40abb40 0x7f5fa41485a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.952+0000 7f5fa8c59640 1 -- 172.21.15.131:0/4202070596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5fa4156930 msgr2=0x7f5fa414c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:29.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.952+0000 7f5fa8c59640 1 --2- 172.21.15.131:0/4202070596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5fa4156930 0x7f5fa414c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:29.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.952+0000 7f5fa8c59640 1 -- 172.21.15.131:0/4202070596 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9c0098e0 con 0x7f5fa40c9370 2023-09-12T14:56:29.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.952+0000 7f5fa8c59640 1 --2- 172.21.15.131:0/4202070596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fa40c9370 0x7f5fa414bf00 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400c4c0 tx=0x7f5f94001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:29.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.952+0000 7f5f99ffb640 1 -- 172.21.15.131:0/4202070596 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f94005480 con 0x7f5fa40c9370 2023-09-12T14:56:29.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.952+0000 7f5f99ffb640 1 -- 172.21.15.131:0/4202070596 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5f94003600 con 0x7f5fa40c9370 2023-09-12T14:56:29.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.952+0000 7f5fab6be640 1 -- 172.21.15.131:0/4202070596 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa41f5070 con 0x7f5fa40c9370 2023-09-12T14:56:29.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.952+0000 7f5f99ffb640 1 -- 172.21.15.131:0/4202070596 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f94003a30 con 0x7f5fa40c9370 2023-09-12T14:56:29.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.952+0000 7f5fab6be640 1 -- 172.21.15.131:0/4202070596 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa406f1e0 con 0x7f5fa40c9370 2023-09-12T14:56:29.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.952+0000 7f5fab6be640 1 -- 172.21.15.131:0/4202070596 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa4073580 con 0x7f5fa40c9370 2023-09-12T14:56:29.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.952+0000 7f5f99ffb640 1 -- 172.21.15.131:0/4202070596 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f5f94003b90 con 0x7f5fa40c9370 2023-09-12T14:56:29.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.956+0000 7f5f99ffb640 1 --2- 172.21.15.131:0/4202070596 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5f88065b20 0x7f5f88067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:29.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.956+0000 7f5f99ffb640 1 -- 172.21.15.131:0/4202070596 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5f94082510 con 0x7f5fa40c9370 2023-09-12T14:56:29.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.956+0000 7f5fa945a640 1 --2- 172.21.15.131:0/4202070596 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5f88065b20 0x7f5f88067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:29.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.956+0000 7f5fa945a640 1 --2- 172.21.15.131:0/4202070596 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5f88065b20 0x7f5f88067fe0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000a790 tx=0x7f5fa0008040 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:29.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:29.960+0000 7f5f99ffb640 1 -- 172.21.15.131:0/4202070596 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f5f94050050 con 0x7f5fa40c9370 2023-09-12T14:56:30.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.104+0000 7f5fab6be640 1 -- 172.21.15.131:0/4202070596 --> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f5fa4165cd0 con 0x7f5f88065b20 2023-09-12T14:56:30.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.104+0000 7f5f99ffb640 1 -- 172.21.15.131:0/4202070596 <== mgr.4200 v2:172.21.15.131:6832/2298103651 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5fa000a430 con 0x7f5f88065b20 2023-09-12T14:56:30.106 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 7 to deep-scrub 2023-09-12T14:56:30.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.104+0000 7f5fab6be640 1 -- 172.21.15.131:0/4202070596 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5f88065b20 msgr2=0x7f5f88067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:30.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.104+0000 7f5fab6be640 1 --2- 172.21.15.131:0/4202070596 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5f88065b20 0x7f5f88067fe0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000a790 tx=0x7f5fa0008040 comp rx=0 tx=0).stop 2023-09-12T14:56:30.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.104+0000 7f5fab6be640 1 -- 172.21.15.131:0/4202070596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fa40c9370 msgr2=0x7f5fa414bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:30.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.104+0000 7f5fab6be640 1 --2- 172.21.15.131:0/4202070596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fa40c9370 0x7f5fa414bf00 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400c4c0 tx=0x7f5f94001e80 comp rx=0 tx=0).stop 2023-09-12T14:56:30.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.104+0000 7f5fab6be640 1 -- 172.21.15.131:0/4202070596 shutdown_connections 2023-09-12T14:56:30.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.104+0000 7f5fab6be640 1 --2- 172.21.15.131:0/4202070596 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5f88065b20 0x7f5f88067fe0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:30.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.104+0000 7f5fab6be640 1 --2- 172.21.15.131:0/4202070596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5fa4156930 0x7f5fa414c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:30.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.104+0000 7f5fab6be640 1 --2- 172.21.15.131:0/4202070596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fa40c9370 0x7f5fa414bf00 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:30.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.104+0000 7f5fab6be640 1 --2- 172.21.15.131:0/4202070596 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5fa40abb40 0x7f5fa41485a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:30.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.104+0000 7f5fab6be640 1 -- 172.21.15.131:0/4202070596 >> 172.21.15.131:0/4202070596 conn(0x7f5fa4069840 msgr2=0x7f5fa415aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:30.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.104+0000 7f5fab6be640 1 -- 172.21.15.131:0/4202070596 shutdown_connections 2023-09-12T14:56:30.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.104+0000 7f5fab6be640 1 -- 172.21.15.131:0/4202070596 wait complete. 2023-09-12T14:56:30.123 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-12T14:56:30.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa036f7c640 1 -- 172.21.15.131:0/2752419540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa018003980 msgr2=0x7fa018003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:30.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa036f7c640 1 --2- 172.21.15.131:0/2752419540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa018003980 0x7fa018003da0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa028009e60 tx=0x7fa02801d130 comp rx=0 tx=0).stop 2023-09-12T14:56:30.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa035519640 1 -- 172.21.15.131:0/2752419540 reap_dead start 2023-09-12T14:56:30.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa036f7c640 1 -- 172.21.15.131:0/2752419540 shutdown_connections 2023-09-12T14:56:30.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa036f7c640 1 -- 172.21.15.131:0/2752419540 >> 172.21.15.131:0/2752419540 conn(0x7fa03006a520 msgr2=0x7fa03006a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:30.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa036f7c640 1 -- 172.21.15.131:0/2752419540 shutdown_connections 2023-09-12T14:56:30.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa036f7c640 1 -- 172.21.15.131:0/2752419540 wait complete. 2023-09-12T14:56:30.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa036f7c640 1 Processor -- start 2023-09-12T14:56:30.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa036f7c640 1 -- start start 2023-09-12T14:56:30.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa036f7c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa018003980 0x7fa0300c4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:30.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa036f7c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0300c2930 0x7fa030142320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:30.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa036f7c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa030153fd0 0x7fa030154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:30.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa036f7c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa030154990 con 0x7fa030153fd0 2023-09-12T14:56:30.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa036f7c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0300c46b0 con 0x7fa0300c2930 2023-09-12T14:56:30.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa036f7c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0300c30d0 con 0x7fa018003980 2023-09-12T14:56:30.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa035519640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa030153fd0 0x7fa030154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:30.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa035519640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa030153fd0 0x7fa030154450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40604/0 (socket says 172.21.15.131:40604) 2023-09-12T14:56:30.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa035519640 1 -- 172.21.15.131:0/55719911 learned_addr learned my addr 172.21.15.131:0/55719911 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:30.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa034d18640 1 --2- 172.21.15.131:0/55719911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa018003980 0x7fa0300c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:30.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa02ffff640 1 --2- 172.21.15.131:0/55719911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0300c2930 0x7fa030142320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:30.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa035519640 1 -- 172.21.15.131:0/55719911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa018003980 msgr2=0x7fa0300c4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:30.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa035519640 1 --2- 172.21.15.131:0/55719911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa018003980 0x7fa0300c4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:30.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa035519640 1 -- 172.21.15.131:0/55719911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0300c2930 msgr2=0x7fa030142320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:30.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa035519640 1 --2- 172.21.15.131:0/55719911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0300c2930 0x7fa030142320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:30.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa035519640 1 -- 172.21.15.131:0/55719911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0300c33b0 con 0x7fa030153fd0 2023-09-12T14:56:30.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa035519640 1 --2- 172.21.15.131:0/55719911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa030153fd0 0x7fa030154450 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa0300cb450 tx=0x7fa024009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:30.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa02d7fa640 1 -- 172.21.15.131:0/55719911 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa024018070 con 0x7fa030153fd0 2023-09-12T14:56:30.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa02d7fa640 1 -- 172.21.15.131:0/55719911 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa024005110 con 0x7fa030153fd0 2023-09-12T14:56:30.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa036f7c640 1 -- 172.21.15.131:0/55719911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0300c3690 con 0x7fa030153fd0 2023-09-12T14:56:30.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa02d7fa640 1 -- 172.21.15.131:0/55719911 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa024005520 con 0x7fa030153fd0 2023-09-12T14:56:30.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.232+0000 7fa036f7c640 1 -- 172.21.15.131:0/55719911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0300c3c50 con 0x7fa030153fd0 2023-09-12T14:56:30.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.236+0000 7fa036f7c640 1 -- 172.21.15.131:0/55719911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa03006d720 con 0x7fa030153fd0 2023-09-12T14:56:30.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.236+0000 7fa02d7fa640 1 -- 172.21.15.131:0/55719911 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fa0240047b0 con 0x7fa030153fd0 2023-09-12T14:56:30.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.236+0000 7fa02d7fa640 1 --2- 172.21.15.131:0/55719911 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa01c065b20 0x7fa01c067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:30.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.236+0000 7fa034d18640 1 --2- 172.21.15.131:0/55719911 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa01c065b20 0x7fa01c067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:30.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.236+0000 7fa02d7fa640 1 -- 172.21.15.131:0/55719911 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa024082db0 con 0x7fa030153fd0 2023-09-12T14:56:30.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.236+0000 7fa034d18640 1 --2- 172.21.15.131:0/55719911 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa01c065b20 0x7fa01c067fe0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa0280ae000 tx=0x7fa028019ed0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:30.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.240+0000 7fa02d7fa640 1 -- 172.21.15.131:0/55719911 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fa024050970 con 0x7fa030153fd0 2023-09-12T14:56:30.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.380+0000 7fa036f7c640 1 -- 172.21.15.131:0/55719911 --> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa030073580 con 0x7fa01c065b20 2023-09-12T14:56:30.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.384+0000 7fa02d7fa640 1 -- 172.21.15.131:0/55719911 <== mgr.4200 v2:172.21.15.131:6832/2298103651 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49483 (secure 0 0 0) 0x7fa030073580 con 0x7fa01c065b20 2023-09-12T14:56:30.389 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:56:30.389 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-09-12T14:56:30.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.388+0000 7fa036f7c640 1 -- 172.21.15.131:0/55719911 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa01c065b20 msgr2=0x7fa01c067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:30.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.388+0000 7fa036f7c640 1 --2- 172.21.15.131:0/55719911 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa01c065b20 0x7fa01c067fe0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa0280ae000 tx=0x7fa028019ed0 comp rx=0 tx=0).stop 2023-09-12T14:56:30.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.388+0000 7fa036f7c640 1 -- 172.21.15.131:0/55719911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa030153fd0 msgr2=0x7fa030154450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:30.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.388+0000 7fa036f7c640 1 --2- 172.21.15.131:0/55719911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa030153fd0 0x7fa030154450 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa0300cb450 tx=0x7fa024009f90 comp rx=0 tx=0).stop 2023-09-12T14:56:30.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.388+0000 7fa036f7c640 1 -- 172.21.15.131:0/55719911 shutdown_connections 2023-09-12T14:56:30.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.388+0000 7fa036f7c640 1 --2- 172.21.15.131:0/55719911 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fa01c065b20 0x7fa01c067fe0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:30.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.388+0000 7fa036f7c640 1 --2- 172.21.15.131:0/55719911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa030153fd0 0x7fa030154450 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:30.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.388+0000 7fa036f7c640 1 --2- 172.21.15.131:0/55719911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0300c2930 0x7fa030142320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:30.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.388+0000 7fa036f7c640 1 --2- 172.21.15.131:0/55719911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa018003980 0x7fa0300c4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:30.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.388+0000 7fa036f7c640 1 -- 172.21.15.131:0/55719911 >> 172.21.15.131:0/55719911 conn(0x7fa03006a520 msgr2=0x7fa0300b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:30.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.388+0000 7fa036f7c640 1 -- 172.21.15.131:0/55719911 shutdown_connections 2023-09-12T14:56:30.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:30.388+0000 7fa036f7c640 1 -- 172.21.15.131:0/55719911 wait complete. 2023-09-12T14:56:30.406 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2023-09-12T14:56:29.604581+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":49,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":224744,"kb_used_data":4420,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749976,"statfs":{"total":773094113280,"available":772863975424,"internally_reserved":0,"allocated":4526080,"data_stored":2496333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":225496632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.322689"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":179,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-09-12T14:56:22.284470+0000","last_change":"2023-09-12T14:54:46.965446+0000","last_active":"2023-09-12T14:56:22.284470+0000","last_peered":"2023-09-12T14:56:22.284470+0000","last_clean":"2023-09-12T14:56:22.284470+0000","last_became_active":"2023-09-12T14:54:46.963236+0000","last_became_peered":"2023-09-12T14:54:46.963236+0000","last_unstale":"2023-09-12T14:56:22.284470+0000","last_undegraded":"2023-09-12T14:56:22.284470+0000","last_fullsized":"2023-09-12T14:56:22.284470+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-12T14:54:45.804232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-12T14:54:45.804232+0000","last_clean_scrub_stamp":"2023-09-12T14:54:45.804232+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T20:24:45.670513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1011712,"data_stored":754945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":1.04},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":1.1240000000000001}]},{"osd":1,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":1.1990000000000001},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":1.256}]},{"osd":2,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.0940000000000001}]},{"osd":3,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.89400000000000002}]},{"osd":4,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.0680000000000001}]},{"osd":5,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.73699999999999999},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.335}]},{"osd":6,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.2290000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.874},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.98899999999999999}]},{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.76900000000000002}]},{"osd":2,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.59199999999999997}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.50800000000000001}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.96399999999999997}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":1.026}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.054},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":1.073}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":164577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":1.085},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.89600000000000002}]},{"osd":1,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.97799999999999998}]},{"osd":2,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.153},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.78200000000000003}]},{"osd":3,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":1.05}]},{"osd":4,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.0169999999999999}]},{"osd":6,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.516,"5min":2.516,"15min":2.516},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.92700000000000005}]},{"osd":7,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":1.21}]}]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":417792,"data_stored":164577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":1.143}]},{"osd":1,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":1.1120000000000001}]},{"osd":2,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":0.76100000000000001}]},{"osd":3,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":1.2809999999999999}]},{"osd":5,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":1.323}]},{"osd":6,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":1.1839999999999999}]},{"osd":7,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":1.246},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":1.3580000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":164577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.58199999999999996}]},{"osd":1,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.64800000000000002}]},{"osd":2,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.85199999999999998}]},{"osd":4,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":1.0009999999999999}]},{"osd":5,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":1.075},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.748}]},{"osd":6,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.91700000000000004}]},{"osd":7,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.78400000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.52800000000000002}]},{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.48299999999999998}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.50900000000000001}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.70899999999999996}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.90400000000000003}]},{"osd":6,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.1989999999999998,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":0.79800000000000004}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.878},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.73199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.49399999999999999}]},{"osd":2,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.60199999999999998}]},{"osd":3,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.56899999999999995}]},{"osd":4,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.75700000000000001}]},{"osd":5,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.79100000000000004}]},{"osd":6,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.88700000000000001}]},{"osd":7,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.68500000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":1007616,"data_stored":753926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.53700000000000003}]},{"osd":2,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.747},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.61899999999999999}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.80400000000000005}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.95499999999999996}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":0.98899999999999999}]},{"osd":6,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.0209999999999999}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.0960000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-12T14:56:30.410 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-12T14:54:45.804232+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=12, tm_hour=14, tm_min=54, tm_sec=45, tm_wday=1, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=12, tm_hour=14, tm_min=56, tm_sec=25, tm_wday=1, tm_yday=255, tm_isdst=0) 2023-09-12T14:56:30.410 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-12T14:56:50.411 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-12T14:56:50.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.543+0000 7f6505bb4640 1 -- 172.21.15.131:0/3782029046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f650006c060 msgr2=0x7f65001626d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:50.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.543+0000 7f6505bb4640 1 --2- 172.21.15.131:0/3782029046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f650006c060 0x7f65001626d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009bc0 tx=0x7f64f801cac0 comp rx=0 tx=0).stop 2023-09-12T14:56:50.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f6505bb4640 1 -- 172.21.15.131:0/3782029046 shutdown_connections 2023-09-12T14:56:50.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f6505bb4640 1 --2- 172.21.15.131:0/3782029046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f650006c060 0x7f65001626d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:50.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f6505bb4640 1 -- 172.21.15.131:0/3782029046 >> 172.21.15.131:0/3782029046 conn(0x7f6500069840 msgr2=0x7f65000cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:50.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f6505bb4640 1 -- 172.21.15.131:0/3782029046 shutdown_connections 2023-09-12T14:56:50.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f6505bb4640 1 -- 172.21.15.131:0/3782029046 wait complete. 2023-09-12T14:56:50.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f6505bb4640 1 Processor -- start 2023-09-12T14:56:50.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f6505bb4640 1 -- start start 2023-09-12T14:56:50.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f6505bb4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f650006c060 0x7f65000c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:50.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f6505bb4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65000ab730 0x7f650014f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:50.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f6505bb4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6500158180 0x7f650014fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:50.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f6505bb4640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6500146bc0 con 0x7f6500158180 2023-09-12T14:56:50.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f6505bb4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65001503e0 con 0x7f65000ab730 2023-09-12T14:56:50.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f6505bb4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65001506c0 con 0x7f650006c060 2023-09-12T14:56:50.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f64fffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6500158180 0x7f650014fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:50.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f64fffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6500158180 0x7f650014fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37808/0 (socket says 172.21.15.131:37808) 2023-09-12T14:56:50.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f64fffff640 1 -- 172.21.15.131:0/3884715382 learned_addr learned my addr 172.21.15.131:0/3884715382 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:50.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f64fffff640 1 -- 172.21.15.131:0/3884715382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f650006c060 msgr2=0x7f65000c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:50.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f64fffff640 1 --2- 172.21.15.131:0/3884715382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f650006c060 0x7f65000c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:50.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f64feffd640 1 --2- 172.21.15.131:0/3884715382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65000ab730 0x7f650014f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:50.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f64fffff640 1 -- 172.21.15.131:0/3884715382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65000ab730 msgr2=0x7f650014f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:50.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f64fffff640 1 --2- 172.21.15.131:0/3884715382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65000ab730 0x7f650014f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:50.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f64fffff640 1 -- 172.21.15.131:0/3884715382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f8009820 con 0x7f6500158180 2023-09-12T14:56:50.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f64feffd640 1 --2- 172.21.15.131:0/3884715382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65000ab730 0x7f650014f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:50.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f64fffff640 1 --2- 172.21.15.131:0/3884715382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6500158180 0x7f650014fd80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009bc0 tx=0x7f64f8003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:50.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f64e7fff640 1 -- 172.21.15.131:0/3884715382 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f80a5bb0 con 0x7f6500158180 2023-09-12T14:56:50.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f64e7fff640 1 -- 172.21.15.131:0/3884715382 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f64f8004130 con 0x7f6500158180 2023-09-12T14:56:50.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f6505bb4640 1 -- 172.21.15.131:0/3884715382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6500146dd0 con 0x7f6500158180 2023-09-12T14:56:50.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f64e7fff640 1 -- 172.21.15.131:0/3884715382 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f8004500 con 0x7f6500158180 2023-09-12T14:56:50.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.547+0000 7f6505bb4640 1 -- 172.21.15.131:0/3884715382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650014d3c0 con 0x7f6500158180 2023-09-12T14:56:50.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.551+0000 7f6505bb4640 1 -- 172.21.15.131:0/3884715382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6500073580 con 0x7f6500158180 2023-09-12T14:56:50.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.551+0000 7f64e7fff640 1 -- 172.21.15.131:0/3884715382 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f64f8004660 con 0x7f6500158180 2023-09-12T14:56:50.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.551+0000 7f64e7fff640 1 --2- 172.21.15.131:0/3884715382 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f64ec065b20 0x7f64ec067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:50.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.551+0000 7f64e7fff640 1 -- 172.21.15.131:0/3884715382 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f64f811e0e0 con 0x7f6500158180 2023-09-12T14:56:50.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.551+0000 7f64ff7fe640 1 --2- 172.21.15.131:0/3884715382 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f64ec065b20 0x7f64ec067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:50.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.551+0000 7f64ff7fe640 1 --2- 172.21.15.131:0/3884715382 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f64ec065b20 0x7f64ec067fe0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f64f0001600 tx=0x7f64f0007930 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:50.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.555+0000 7f64e7fff640 1 -- 172.21.15.131:0/3884715382 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f64f80ebd30 con 0x7f6500158180 2023-09-12T14:56:50.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.699+0000 7f6505bb4640 1 -- 172.21.15.131:0/3884715382 --> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6500147280 con 0x7f64ec065b20 2023-09-12T14:56:50.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.703+0000 7f64e7fff640 1 -- 172.21.15.131:0/3884715382 <== mgr.4200 v2:172.21.15.131:6832/2298103651 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49247 (secure 0 0 0) 0x7f64f000c480 con 0x7f64ec065b20 2023-09-12T14:56:50.709 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:56:50.709 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-09-12T14:56:50.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.707+0000 7f6505bb4640 1 -- 172.21.15.131:0/3884715382 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f64ec065b20 msgr2=0x7f64ec067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:50.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.707+0000 7f6505bb4640 1 --2- 172.21.15.131:0/3884715382 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f64ec065b20 0x7f64ec067fe0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f64f0001600 tx=0x7f64f0007930 comp rx=0 tx=0).stop 2023-09-12T14:56:50.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.707+0000 7f6505bb4640 1 -- 172.21.15.131:0/3884715382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6500158180 msgr2=0x7f650014fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:50.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.707+0000 7f6505bb4640 1 --2- 172.21.15.131:0/3884715382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6500158180 0x7f650014fd80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009bc0 tx=0x7f64f8003f40 comp rx=0 tx=0).stop 2023-09-12T14:56:50.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.707+0000 7f6505bb4640 1 -- 172.21.15.131:0/3884715382 shutdown_connections 2023-09-12T14:56:50.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.707+0000 7f6505bb4640 1 --2- 172.21.15.131:0/3884715382 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f64ec065b20 0x7f64ec067fe0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:50.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.707+0000 7f6505bb4640 1 --2- 172.21.15.131:0/3884715382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6500158180 0x7f650014fd80 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:50.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.707+0000 7f6505bb4640 1 --2- 172.21.15.131:0/3884715382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65000ab730 0x7f650014f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:50.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.707+0000 7f6505bb4640 1 --2- 172.21.15.131:0/3884715382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f650006c060 0x7f65000c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:50.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.707+0000 7f6505bb4640 1 -- 172.21.15.131:0/3884715382 >> 172.21.15.131:0/3884715382 conn(0x7f6500069840 msgr2=0x7f65000cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:50.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.707+0000 7f6505bb4640 1 -- 172.21.15.131:0/3884715382 shutdown_connections 2023-09-12T14:56:50.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.707+0000 7f6505bb4640 1 -- 172.21.15.131:0/3884715382 wait complete. 2023-09-12T14:56:50.726 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2023-09-12T14:56:49.609701+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":224736,"kb_used_data":4420,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749984,"statfs":{"total":773094113280,"available":772863983616,"internally_reserved":0,"allocated":4526080,"data_stored":2496333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":225496632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003068"},"pg_stats":[{"pgid":"1.0","version":"36'119","reported_seq":190,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-09-12T14:56:30.132990+0000","last_change":"2023-09-12T14:56:30.133136+0000","last_active":"2023-09-12T14:56:30.132990+0000","last_peered":"2023-09-12T14:56:30.132990+0000","last_clean":"2023-09-12T14:56:30.132990+0000","last_became_active":"2023-09-12T14:54:46.963236+0000","last_became_peered":"2023-09-12T14:54:46.963236+0000","last_unstale":"2023-09-12T14:56:30.132990+0000","last_undegraded":"2023-09-12T14:56:30.132990+0000","last_fullsized":"2023-09-12T14:56:30.132990+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'119","last_scrub_stamp":"2023-09-12T14:56:30.132765+0000","last_deep_scrub":"36'119","last_deep_scrub_stamp":"2023-09-12T14:56:30.132765+0000","last_clean_scrub_stamp":"2023-09-12T14:56:30.132765+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-13T23:41:24.058052+0000","scrub_duration":0.010430241999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1011712,"data_stored":754945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":1.117}]},{"osd":1,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.878},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":1.0089999999999999}]},{"osd":2,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.161},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.78600000000000003}]},{"osd":3,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.073},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":1.044}]},{"osd":4,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.90900000000000003}]},{"osd":5,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.1970000000000001}]},{"osd":6,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.236}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":1.01},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.84299999999999997}]},{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.117},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":1.137}]},{"osd":2,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.746},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":1.2010000000000001}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.88400000000000001}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.92700000000000005}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":1.242}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.093},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":1.1559999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":164577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.88600000000000001}]},{"osd":1,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.755}]},{"osd":2,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.94599999999999995}]},{"osd":3,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":1.224}]},{"osd":4,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.1850000000000001}]},{"osd":6,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.516,"5min":2.516,"15min":2.516},"last":1.147},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":1.077}]},{"osd":7,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.109},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":1.2809999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":417792,"data_stored":164577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.90000000000000002}]},{"osd":1,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":1.1319999999999999},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":1.026}]},{"osd":2,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.75},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":1.1639999999999999}]},{"osd":3,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.83899999999999997}]},{"osd":5,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.25},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.79200000000000004}]},{"osd":6,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":1.0980000000000001}]},{"osd":7,"last update":"Tue Sep 12 14:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.95999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":164577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.75900000000000001}]},{"osd":1,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.83199999999999996}]},{"osd":2,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":1.038}]},{"osd":4,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.879},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":1.1499999999999999}]},{"osd":5,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.94199999999999995}]},{"osd":6,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.014},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.71999999999999997}]},{"osd":7,"last update":"Tue Sep 12 14:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.96699999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57399999999999995,"15min":0.57199999999999995},"min":{"1min":0.28699999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.98999999999999999,"5min":1.046,"15min":1.046},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.68600000000000005,"15min":0.69199999999999995},"min":{"1min":0.30399999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0800000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.57699999999999996}]},{"osd":1,"last update":"Tue Sep 12 14:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.64300000000000002,"15min":0.64400000000000002},"min":{"1min":0.34799999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.99099999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66300000000000003,"15min":0.65900000000000003},"min":{"1min":0.39700000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.0549999999999999,"5min":1.093,"15min":1.093},"last":0.874}]},{"osd":3,"last update":"Tue Sep 12 14:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.29999999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.107,"5min":1.119,"15min":1.119},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.78000000000000003,"15min":0.78200000000000003},"min":{"1min":0.41899999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.2010000000000001,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.095}]},{"osd":4,"last update":"Tue Sep 12 14:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.93899999999999995,"15min":0.94299999999999995},"min":{"1min":0.56100000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.254,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.2010000000000001},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.998,"15min":1.0089999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.534,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.77300000000000002}]},{"osd":5,"last update":"Tue Sep 12 14:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.96399999999999997,"15min":0.97099999999999997},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.93300000000000005,"15min":0.93200000000000005},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.304,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.1279999999999999}]},{"osd":6,"last update":"Tue Sep 12 14:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.93799999999999994,"15min":0.94199999999999995},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.3169999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.04,"15min":1.0489999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.304,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":1.1559999999999999}]},{"osd":7,"last update":"Tue Sep 12 14:56:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0449999999999999,"15min":1.0509999999999999},"min":{"1min":0.66600000000000004,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":1.024},{"interface":"front","average":{"1min":0.98899999999999999,"5min":1.0089999999999999,"15min":1.0129999999999999},"min":{"1min":0.63500000000000001,"5min":0.627,"15min":0.627},"max":{"1min":1.2709999999999999,"5min":1.915,"15min":1.915},"last":1.054}]}]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.57999999999999996}]},{"osd":2,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.86799999999999999}]},{"osd":3,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":1.0249999999999999}]},{"osd":4,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":1.071},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.052}]},{"osd":5,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.94899999999999995}]},{"osd":6,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":1.143}]},{"osd":7,"last update":"Tue Sep 12 14:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":1.1080000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.98199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":1007616,"data_stored":753926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.54700000000000004}]},{"osd":2,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.63800000000000001}]},{"osd":3,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.86099999999999999}]},{"osd":4,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.1759999999999999},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.97399999999999998}]},{"osd":5,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":0.93700000000000006}]},{"osd":6,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.0880000000000001}]},{"osd":7,"last update":"Tue Sep 12 14:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.90200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-12T14:56:50.728 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-09-12T14:56:50.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f56a359b640 1 -- 172.21.15.131:0/3984395291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f569c0abb00 msgr2=0x7f569c0ac050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:50.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f56a359b640 1 --2- 172.21.15.131:0/3984395291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f569c0abb00 0x7f569c0ac050 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5694009840 tx=0x7f569401c770 comp rx=0 tx=0).stop 2023-09-12T14:56:50.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f56a359b640 1 -- 172.21.15.131:0/3984395291 shutdown_connections 2023-09-12T14:56:50.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f56a359b640 1 --2- 172.21.15.131:0/3984395291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f569c0abb00 0x7f569c0ac050 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:50.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f56a359b640 1 -- 172.21.15.131:0/3984395291 >> 172.21.15.131:0/3984395291 conn(0x7f569c069840 msgr2=0x7f569c0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:50.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f56a359b640 1 -- 172.21.15.131:0/3984395291 shutdown_connections 2023-09-12T14:56:50.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f56a359b640 1 -- 172.21.15.131:0/3984395291 wait complete. 2023-09-12T14:56:50.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f56a359b640 1 Processor -- start 2023-09-12T14:56:50.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f56a359b640 1 -- start start 2023-09-12T14:56:50.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f56a359b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f569c06f3b0 0x7f569c1547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:50.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f56a359b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f569c0abb00 0x7f569c145e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:50.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f56a359b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f569c1572c0 0x7f569c13fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:50.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f56a359b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c153790 con 0x7f569c1572c0 2023-09-12T14:56:50.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f56a359b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c1539e0 con 0x7f569c0abb00 2023-09-12T14:56:50.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f56a359b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569c153cc0 con 0x7f569c06f3b0 2023-09-12T14:56:50.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f56a1b38640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f569c1572c0 0x7f569c13fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:50.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f56a1b38640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f569c1572c0 0x7f569c13fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37826/0 (socket says 172.21.15.131:37826) 2023-09-12T14:56:50.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f56a1b38640 1 -- 172.21.15.131:0/2832940936 learned_addr learned my addr 172.21.15.131:0/2832940936 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:50.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f56a1b38640 1 -- 172.21.15.131:0/2832940936 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f569c06f3b0 msgr2=0x7f569c1547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:50.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f56a1b38640 1 --2- 172.21.15.131:0/2832940936 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f569c06f3b0 0x7f569c1547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:50.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f56a1b38640 1 -- 172.21.15.131:0/2832940936 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f569c0abb00 msgr2=0x7f569c145e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:50.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f56a1b38640 1 --2- 172.21.15.131:0/2832940936 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f569c0abb00 0x7f569c145e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:50.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f56a1b38640 1 -- 172.21.15.131:0/2832940936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56940094a0 con 0x7f569c1572c0 2023-09-12T14:56:50.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f56a1b38640 1 --2- 172.21.15.131:0/2832940936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f569c1572c0 0x7f569c13fef0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5694009790 tx=0x7f56940a67b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:50.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f5699ffb640 1 -- 172.21.15.131:0/2832940936 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56940042c0 con 0x7f569c1572c0 2023-09-12T14:56:50.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f5699ffb640 1 -- 172.21.15.131:0/2832940936 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5694004510 con 0x7f569c1572c0 2023-09-12T14:56:50.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f56a359b640 1 -- 172.21.15.131:0/2832940936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f569c153fa0 con 0x7f569c1572c0 2023-09-12T14:56:50.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f5699ffb640 1 -- 172.21.15.131:0/2832940936 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5694003800 con 0x7f569c1572c0 2023-09-12T14:56:50.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.839+0000 7f56a359b640 1 -- 172.21.15.131:0/2832940936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f569c1463a0 con 0x7f569c1572c0 2023-09-12T14:56:50.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.843+0000 7f56a359b640 1 -- 172.21.15.131:0/2832940936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f569c073580 con 0x7f569c1572c0 2023-09-12T14:56:50.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.843+0000 7f5699ffb640 1 -- 172.21.15.131:0/2832940936 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7f5694003960 con 0x7f569c1572c0 2023-09-12T14:56:50.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.843+0000 7f5699ffb640 1 --2- 172.21.15.131:0/2832940936 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5688065b20 0x7f5688067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:50.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.843+0000 7f5699ffb640 1 -- 172.21.15.131:0/2832940936 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f56940aa070 con 0x7f569c1572c0 2023-09-12T14:56:50.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.843+0000 7f56a1337640 1 --2- 172.21.15.131:0/2832940936 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5688065b20 0x7f5688067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:50.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.843+0000 7f56a1337640 1 --2- 172.21.15.131:0/2832940936 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5688065b20 0x7f5688067fe0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f568c001560 tx=0x7f568c007930 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:50.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.847+0000 7f5699ffb640 1 -- 172.21.15.131:0/2832940936 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7f56940ec420 con 0x7f569c1572c0 2023-09-12T14:56:50.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:50.987+0000 7f56a359b640 1 -- 172.21.15.131:0/2832940936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f569c154450 con 0x7f569c1572c0 2023-09-12T14:56:51.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.003+0000 7f5699ffb640 1 -- 172.21.15.131:0/2832940936 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v33)=0 v33) v1 ==== 123+0+0 (secure 0 0 0) 0x7f56940b7e60 con 0x7f569c1572c0 2023-09-12T14:56:51.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.003+0000 7f5699ffb640 1 -- 172.21.15.131:0/2832940936 <== mon.0 v2:172.21.15.131:3300/0 8 ==== config(6 keys) v1 ==== 172+0+0 (secure 0 0 0) 0x7f56940ec420 con 0x7f569c1572c0 2023-09-12T14:56:51.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.007+0000 7f56a359b640 1 -- 172.21.15.131:0/2832940936 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5688065b20 msgr2=0x7f5688067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:51.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.007+0000 7f56a359b640 1 --2- 172.21.15.131:0/2832940936 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5688065b20 0x7f5688067fe0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f568c001560 tx=0x7f568c007930 comp rx=0 tx=0).stop 2023-09-12T14:56:51.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.007+0000 7f56a359b640 1 -- 172.21.15.131:0/2832940936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f569c1572c0 msgr2=0x7f569c13fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:51.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.007+0000 7f56a359b640 1 --2- 172.21.15.131:0/2832940936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f569c1572c0 0x7f569c13fef0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5694009790 tx=0x7f56940a67b0 comp rx=0 tx=0).stop 2023-09-12T14:56:51.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.007+0000 7f56a359b640 1 -- 172.21.15.131:0/2832940936 shutdown_connections 2023-09-12T14:56:51.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.007+0000 7f56a359b640 1 --2- 172.21.15.131:0/2832940936 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7f5688065b20 0x7f5688067fe0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:51.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.007+0000 7f56a359b640 1 --2- 172.21.15.131:0/2832940936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f569c1572c0 0x7f569c13fef0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:51.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.007+0000 7f56a359b640 1 --2- 172.21.15.131:0/2832940936 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f569c0abb00 0x7f569c145e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:51.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.007+0000 7f56a359b640 1 --2- 172.21.15.131:0/2832940936 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f569c06f3b0 0x7f569c1547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:51.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.007+0000 7f56a359b640 1 -- 172.21.15.131:0/2832940936 >> 172.21.15.131:0/2832940936 conn(0x7f569c069840 msgr2=0x7f569c0cc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:51.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.007+0000 7f56a359b640 1 -- 172.21.15.131:0/2832940936 shutdown_connections 2023-09-12T14:56:51.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.007+0000 7f56a359b640 1 -- 172.21.15.131:0/2832940936 wait complete. 2023-09-12T14:56:51.025 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-12T14:56:51.026 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-12T14:56:51.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.139+0000 7fd3dae1b640 1 -- 172.21.15.131:0/3474310240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3d40abb00 msgr2=0x7fd3d406eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:51.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.139+0000 7fd3dae1b640 1 --2- 172.21.15.131:0/3474310240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3d40abb00 0x7fd3d406eca0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000aa10 tx=0x7fd3c001db40 comp rx=0 tx=0).stop 2023-09-12T14:56:51.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.139+0000 7fd3dae1b640 1 -- 172.21.15.131:0/3474310240 shutdown_connections 2023-09-12T14:56:51.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.139+0000 7fd3dae1b640 1 --2- 172.21.15.131:0/3474310240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3d40abb00 0x7fd3d406eca0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:51.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.139+0000 7fd3dae1b640 1 --2- 172.21.15.131:0/3474310240 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3d41572c0 0x7fd3d40ac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:51.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.139+0000 7fd3dae1b640 1 -- 172.21.15.131:0/3474310240 >> 172.21.15.131:0/3474310240 conn(0x7fd3d4069840 msgr2=0x7fd3d40cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:51.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.139+0000 7fd3dae1b640 1 -- 172.21.15.131:0/3474310240 shutdown_connections 2023-09-12T14:56:51.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.139+0000 7fd3dae1b640 1 -- 172.21.15.131:0/3474310240 wait complete. 2023-09-12T14:56:51.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.143+0000 7fd3dae1b640 1 Processor -- start 2023-09-12T14:56:51.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.143+0000 7fd3dae1b640 1 -- start start 2023-09-12T14:56:51.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.143+0000 7fd3dae1b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3d406f1e0 0x7fd3d40a9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:51.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.143+0000 7fd3dae1b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3d40abb00 0x7fd3d40aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:51.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.143+0000 7fd3dae1b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3d41572c0 0x7fd3d4153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:51.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.143+0000 7fd3dae1b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d4139200 con 0x7fd3d406f1e0 2023-09-12T14:56:51.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.143+0000 7fd3dae1b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d41394e0 con 0x7fd3d41572c0 2023-09-12T14:56:51.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.143+0000 7fd3dae1b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d41397c0 con 0x7fd3d40abb00 2023-09-12T14:56:51.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.143+0000 7fd3d8bb7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3d406f1e0 0x7fd3d40a9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:51.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.143+0000 7fd3d8bb7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3d406f1e0 0x7fd3d40a9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37840/0 (socket says 172.21.15.131:37840) 2023-09-12T14:56:51.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.143+0000 7fd3d8bb7640 1 -- 172.21.15.131:0/24622970 learned_addr learned my addr 172.21.15.131:0/24622970 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-09-12T14:56:51.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.143+0000 7fd3d93b8640 1 --2- 172.21.15.131:0/24622970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3d41572c0 0x7fd3d4153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:51.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.143+0000 7fd3d3fff640 1 --2- 172.21.15.131:0/24622970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3d40abb00 0x7fd3d40aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:51.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.143+0000 7fd3d8bb7640 1 -- 172.21.15.131:0/24622970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3d40abb00 msgr2=0x7fd3d40aadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:51.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.143+0000 7fd3d8bb7640 1 --2- 172.21.15.131:0/24622970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3d40abb00 0x7fd3d40aadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:51.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.143+0000 7fd3d8bb7640 1 -- 172.21.15.131:0/24622970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3d41572c0 msgr2=0x7fd3d4153790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:51.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.143+0000 7fd3d8bb7640 1 --2- 172.21.15.131:0/24622970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3d41572c0 0x7fd3d4153790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:51.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.143+0000 7fd3d8bb7640 1 -- 172.21.15.131:0/24622970 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c000a670 con 0x7fd3d406f1e0 2023-09-12T14:56:51.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.143+0000 7fd3d93b8640 1 --2- 172.21.15.131:0/24622970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3d41572c0 0x7fd3d4153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-12T14:56:51.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.143+0000 7fd3d8bb7640 1 --2- 172.21.15.131:0/24622970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3d406f1e0 0x7fd3d40a9170 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd3c800c9f0 tx=0x7fd3c800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:51.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.143+0000 7fd3d17fa640 1 -- 172.21.15.131:0/24622970 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8007bb0 con 0x7fd3d406f1e0 2023-09-12T14:56:51.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.143+0000 7fd3d17fa640 1 -- 172.21.15.131:0/24622970 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(6 keys) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd3c8007e20 con 0x7fd3d406f1e0 2023-09-12T14:56:51.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.143+0000 7fd3dae1b640 1 -- 172.21.15.131:0/24622970 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3d4139aa0 con 0x7fd3d406f1e0 2023-09-12T14:56:51.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.143+0000 7fd3d17fa640 1 -- 172.21.15.131:0/24622970 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c800f6b0 con 0x7fd3d406f1e0 2023-09-12T14:56:51.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.143+0000 7fd3dae1b640 1 -- 172.21.15.131:0/24622970 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3d40ab330 con 0x7fd3d406f1e0 2023-09-12T14:56:51.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.143+0000 7fd3dae1b640 1 -- 172.21.15.131:0/24622970 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3d4073580 con 0x7fd3d406f1e0 2023-09-12T14:56:51.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.147+0000 7fd3d17fa640 1 -- 172.21.15.131:0/24622970 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 85354+0+0 (secure 0 0 0) 0x7fd3c800f810 con 0x7fd3d406f1e0 2023-09-12T14:56:51.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.147+0000 7fd3d17fa640 1 --2- 172.21.15.131:0/24622970 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd3b8065b20 0x7fd3b8067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-12T14:56:51.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.147+0000 7fd3d3fff640 1 --2- 172.21.15.131:0/24622970 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd3b8065b20 0x7fd3b8067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-12T14:56:51.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.147+0000 7fd3d17fa640 1 -- 172.21.15.131:0/24622970 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd3c8083850 con 0x7fd3d406f1e0 2023-09-12T14:56:51.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.147+0000 7fd3d3fff640 1 --2- 172.21.15.131:0/24622970 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd3b8065b20 0x7fd3b8067fe0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd3c00b0000 tx=0x7fd3c001a8e0 comp rx=0 tx=0).ready entity=mgr.4200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-12T14:56:51.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.151+0000 7fd3d17fa640 1 -- 172.21.15.131:0/24622970 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171412 (secure 0 0 0) 0x7fd3c8051530 con 0x7fd3d406f1e0 2023-09-12T14:56:51.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.339+0000 7fd3dae1b640 1 -- 172.21.15.131:0/24622970 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3d4139c30 con 0x7fd3d406f1e0 2023-09-12T14:56:51.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.339+0000 7fd3d17fa640 1 -- 172.21.15.131:0/24622970 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+3704 (secure 0 0 0) 0x7fd3c801d090 con 0x7fd3d406f1e0 2023-09-12T14:56:51.343 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-09-12T14:56:51.344 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":15,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4676,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/3890238785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3890238785},{"type":"v1","addr":"172.21.15.187:6834","nonce":3890238785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4978,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1436257801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":1436257801},{"type":"v1","addr":"172.21.15.187:6835","nonce":1436257801}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4990,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/806466310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":806466310},{"type":"v1","addr":"172.21.15.131:6837","nonce":806466310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/922195333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":922195333},{"type":"v1","addr":"172.21.15.131:6835","nonce":922195333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15}],"filesystems":[]} 2023-09-12T14:56:51.344 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 15 2023-09-12T14:56:51.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.343+0000 7fd3dae1b640 1 -- 172.21.15.131:0/24622970 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd3b8065b20 msgr2=0x7fd3b8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:51.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.343+0000 7fd3dae1b640 1 --2- 172.21.15.131:0/24622970 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd3b8065b20 0x7fd3b8067fe0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd3c00b0000 tx=0x7fd3c001a8e0 comp rx=0 tx=0).stop 2023-09-12T14:56:51.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.343+0000 7fd3dae1b640 1 -- 172.21.15.131:0/24622970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3d406f1e0 msgr2=0x7fd3d40a9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-12T14:56:51.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.343+0000 7fd3dae1b640 1 --2- 172.21.15.131:0/24622970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3d406f1e0 0x7fd3d40a9170 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd3c800c9f0 tx=0x7fd3c800ceb0 comp rx=0 tx=0).stop 2023-09-12T14:56:51.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.343+0000 7fd3dae1b640 1 -- 172.21.15.131:0/24622970 shutdown_connections 2023-09-12T14:56:51.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.343+0000 7fd3dae1b640 1 --2- 172.21.15.131:0/24622970 >> [v2:172.21.15.131:6832/2298103651,v1:172.21.15.131:6833/2298103651] conn(0x7fd3b8065b20 0x7fd3b8067fe0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:51.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.343+0000 7fd3dae1b640 1 --2- 172.21.15.131:0/24622970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3d40abb00 0x7fd3d40aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:51.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.343+0000 7fd3dae1b640 1 --2- 172.21.15.131:0/24622970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3d41572c0 0x7fd3d4153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:51.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.343+0000 7fd3dae1b640 1 --2- 172.21.15.131:0/24622970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3d406f1e0 0x7fd3d40a9170 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-12T14:56:51.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.343+0000 7fd3dae1b640 1 -- 172.21.15.131:0/24622970 >> 172.21.15.131:0/24622970 conn(0x7fd3d4069840 msgr2=0x7fd3d4159710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-12T14:56:51.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.343+0000 7fd3dae1b640 1 -- 172.21.15.131:0/24622970 shutdown_connections 2023-09-12T14:56:51.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-09-12T14:56:51.343+0000 7fd3dae1b640 1 -- 172.21.15.131:0/24622970 wait complete. 2023-09-12T14:56:51.362 DEBUG:tasks.cephfs.filesystem:already dead... 2023-09-12T14:56:51.363 INFO:teuthology.misc:Shutting down mds daemons... 2023-09-12T14:56:51.363 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-12T14:56:51.364 INFO:teuthology.orchestra.run:waiting for 300 2023-09-12T14:56:51.374 INFO:tasks.ceph.mds.a:Stopped 2023-09-12T14:56:51.375 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-09-12T14:56:51.375 INFO:teuthology.orchestra.run:waiting for 300 2023-09-12T14:56:51.428 INFO:tasks.ceph.mds.c:Stopped 2023-09-12T14:56:51.428 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-09-12T14:56:51.429 INFO:teuthology.orchestra.run:waiting for 300 2023-09-12T14:56:51.447 INFO:tasks.ceph.mds.b:Stopped 2023-09-12T14:56:51.447 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-09-12T14:56:51.448 INFO:teuthology.orchestra.run:waiting for 300 2023-09-12T14:56:51.502 INFO:tasks.ceph.mds.d:Stopped 2023-09-12T14:56:51.503 INFO:teuthology.misc:Shutting down osd daemons... 2023-09-12T14:56:51.503 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-09-12T14:56:51.504 INFO:teuthology.orchestra.run:waiting for 300 2023-09-12T14:56:51.564 INFO:tasks.ceph.osd.0:Stopped 2023-09-12T14:56:51.565 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-09-12T14:56:51.565 INFO:teuthology.orchestra.run:waiting for 300 2023-09-12T14:56:51.639 INFO:tasks.ceph.osd.1:Stopped 2023-09-12T14:56:51.640 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-09-12T14:56:51.640 INFO:teuthology.orchestra.run:waiting for 300 2023-09-12T14:56:51.718 INFO:tasks.ceph.osd.2:Stopped 2023-09-12T14:56:51.719 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-09-12T14:56:51.719 INFO:teuthology.orchestra.run:waiting for 300 2023-09-12T14:56:55.772 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-12T14:56:55.775 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-12T14:56:57.419 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2023-09-12T14:56:57.821 INFO:tasks.ceph.osd.3:Stopped 2023-09-12T14:56:57.822 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-09-12T14:56:57.822 INFO:teuthology.orchestra.run:waiting for 300 2023-09-12T14:56:57.886 INFO:tasks.ceph.osd.4:Stopped 2023-09-12T14:56:57.886 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-09-12T14:56:57.887 INFO:teuthology.orchestra.run:waiting for 300 2023-09-12T14:56:57.986 INFO:tasks.ceph.osd.5:Stopped 2023-09-12T14:56:57.987 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-09-12T14:56:57.987 INFO:teuthology.orchestra.run:waiting for 300 2023-09-12T14:57:03.025 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2023-09-12T14:57:03.025 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2023-09-12T14:57:04.089 INFO:tasks.ceph.osd.6:Stopped 2023-09-12T14:57:04.090 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-09-12T14:57:04.091 INFO:teuthology.orchestra.run:waiting for 300 2023-09-12T14:57:04.171 INFO:tasks.ceph.osd.7:Stopped 2023-09-12T14:57:04.172 INFO:teuthology.misc:Shutting down mgr daemons... 2023-09-12T14:57:04.173 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-09-12T14:57:04.173 INFO:teuthology.orchestra.run:waiting for 300 2023-09-12T14:57:04.204 INFO:tasks.ceph.mgr.y:Stopped 2023-09-12T14:57:04.205 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-09-12T14:57:04.205 INFO:teuthology.orchestra.run:waiting for 300 2023-09-12T14:57:04.231 INFO:tasks.ceph.mgr.x:Stopped 2023-09-12T14:57:04.232 INFO:teuthology.misc:Shutting down mon daemons... 2023-09-12T14:57:04.233 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-09-12T14:57:04.233 INFO:teuthology.orchestra.run:waiting for 300 2023-09-12T14:57:04.260 INFO:tasks.ceph.mon.a:Stopped 2023-09-12T14:57:04.260 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-09-12T14:57:04.261 INFO:teuthology.orchestra.run:waiting for 300 2023-09-12T14:57:04.290 INFO:tasks.ceph.mon.b:Stopped 2023-09-12T14:57:04.290 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-09-12T14:57:04.291 INFO:teuthology.orchestra.run:waiting for 300 2023-09-12T14:57:04.356 INFO:tasks.ceph.mon.c:Stopped 2023-09-12T14:57:04.357 INFO:tasks.ceph:Checking cluster log for badness... 2023-09-12T14:57:04.357 DEBUG:teuthology.orchestra.run.smithi131:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-09-12T14:57:04.375 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi131.front.sepia.ceph.com 2023-09-12T14:57:04.376 DEBUG:teuthology.orchestra.run.smithi131:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-09-12T14:57:04.584 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi131.front.sepia.ceph.com 2023-09-12T14:57:04.584 DEBUG:teuthology.orchestra.run.smithi131:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-09-12T14:57:04.798 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi131.front.sepia.ceph.com 2023-09-12T14:57:04.799 DEBUG:teuthology.orchestra.run.smithi131:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-09-12T14:57:04.982 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi131.front.sepia.ceph.com 2023-09-12T14:57:04.983 DEBUG:teuthology.orchestra.run.smithi131:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-09-12T14:57:05.182 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi187.front.sepia.ceph.com 2023-09-12T14:57:05.183 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-09-12T14:57:05.496 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi187.front.sepia.ceph.com 2023-09-12T14:57:05.496 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-09-12T14:57:05.661 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi187.front.sepia.ceph.com 2023-09-12T14:57:05.662 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-09-12T14:57:05.786 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi187.front.sepia.ceph.com 2023-09-12T14:57:05.786 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-09-12T14:57:05.920 INFO:tasks.ceph:Archiving mon data... 2023-09-12T14:57:05.921 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395049/data/mon.a.tgz 2023-09-12T14:57:05.922 DEBUG:teuthology.orchestra.run.smithi131:> mktemp 2023-09-12T14:57:05.928 INFO:teuthology.orchestra.run.smithi131.stdout:/tmp/tmp.82oYWSWwsC 2023-09-12T14:57:05.929 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.82oYWSWwsC 2023-09-12T14:57:06.042 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0666 /tmp/tmp.82oYWSWwsC 2023-09-12T14:57:06.105 DEBUG:teuthology.orchestra.remote:smithi131:/tmp/tmp.82oYWSWwsC is 291KB 2023-09-12T14:57:06.162 DEBUG:teuthology.orchestra.run.smithi131:> rm -fr /tmp/tmp.82oYWSWwsC 2023-09-12T14:57:06.168 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395049/data/mon.b.tgz 2023-09-12T14:57:06.168 DEBUG:teuthology.orchestra.run.smithi187:> mktemp 2023-09-12T14:57:06.174 INFO:teuthology.orchestra.run.smithi187.stdout:/tmp/tmp.gl1njbh5jp 2023-09-12T14:57:06.174 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.gl1njbh5jp 2023-09-12T14:57:06.307 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0666 /tmp/tmp.gl1njbh5jp 2023-09-12T14:57:06.369 DEBUG:teuthology.orchestra.remote:smithi187:/tmp/tmp.gl1njbh5jp is 389KB 2023-09-12T14:57:06.428 DEBUG:teuthology.orchestra.run.smithi187:> rm -fr /tmp/tmp.gl1njbh5jp 2023-09-12T14:57:06.434 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395049/data/mon.c.tgz 2023-09-12T14:57:06.434 DEBUG:teuthology.orchestra.run.smithi187:> mktemp 2023-09-12T14:57:06.482 INFO:teuthology.orchestra.run.smithi187.stdout:/tmp/tmp.c1fEsOp3gj 2023-09-12T14:57:06.482 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.c1fEsOp3gj 2023-09-12T14:57:06.613 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0666 /tmp/tmp.c1fEsOp3gj 2023-09-12T14:57:06.678 DEBUG:teuthology.orchestra.remote:smithi187:/tmp/tmp.c1fEsOp3gj is 389KB 2023-09-12T14:57:06.780 DEBUG:teuthology.orchestra.run.smithi187:> rm -fr /tmp/tmp.c1fEsOp3gj 2023-09-12T14:57:06.786 INFO:tasks.ceph:Cleaning ceph cluster... 2023-09-12T14:57:06.786 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-09-12T14:57:06.789 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-09-12T14:57:06.842 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-09-12T14:57:06.843 DEBUG:teuthology.orchestra.run.smithi131:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-09-12T14:57:06.846 DEBUG:teuthology.orchestra.run.smithi187:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-09-12T14:57:06.873 INFO:teuthology.orchestra.run.smithi131.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-09-12T14:57:06.910 INFO:teuthology.orchestra.run.smithi187.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-09-12T14:57:06.914 INFO:tasks.ceph:Archiving crash dumps... 2023-09-12T14:57:06.916 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395049/remote/smithi131/crash 2023-09-12T14:57:06.917 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-09-12T14:57:06.933 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395049/remote/smithi187/crash 2023-09-12T14:57:06.934 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-09-12T14:57:06.967 INFO:tasks.ceph:Shutting down logrotate 2023-09-12T14:57:06.968 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-12T14:57:06.978 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-12T14:57:07.050 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-09-12T14:57:07.062 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-09-12T14:57:07.104 INFO:tasks.ceph:Compressing logs... 2023-09-12T14:57:07.105 DEBUG:teuthology.orchestra.run.smithi131:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-12T14:57:07.107 DEBUG:teuthology.orchestra.run.smithi187:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-12T14:57:07.136 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49495.log 2023-09-12T14:57:07.137 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47298.log 2023-09-12T14:57:07.138 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47111.log 2023-09-12T14:57:07.138 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49495.log.gz 2023-09-12T14:57:07.142 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38530.log 2023-09-12T14:57:07.142 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47298.log.gz 2023-09-12T14:57:07.143 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47878.log 2023-09-12T14:57:07.143 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47111.log: /var/log/ceph/ceph-client.admin.38530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50291.log 2023-09-12T14:57:07.143 INFO:teuthology.orchestra.run.smithi131.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.38530.log.gz 2023-09-12T14:57:07.144 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38286.log 2023-09-12T14:57:07.144 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47111.log.gz 2023-09-12T14:57:07.144 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47878.log.gz 2023-09-12T14:57:07.144 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38269.log 2023-09-12T14:57:07.145 INFO:teuthology.orchestra.run.smithi131.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50291.log.gz 2023-09-12T14:57:07.145 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45277.log 2023-09-12T14:57:07.145 INFO:teuthology.orchestra.run.smithi131.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.38286.log.gz 2023-09-12T14:57:07.145 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47142.log 2023-09-12T14:57:07.146 INFO:teuthology.orchestra.run.smithi131.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.38269.log.gz 2023-09-12T14:57:07.146 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-09-12T14:57:07.146 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45277.log: /var/log/ceph/ceph-client.admin.47142.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45277.log.gz 2023-09-12T14:57:07.147 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44856.log 2023-09-12T14:57:07.147 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47142.log.gz 2023-09-12T14:57:07.147 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45510.log 2023-09-12T14:57:07.147 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46347.log 2023-09-12T14:57:07.148 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44856.log.gz 2023-09-12T14:57:07.148 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45536.log 2023-09-12T14:57:07.175 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38165.log 2023-09-12T14:57:07.176 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38189.log 2023-09-12T14:57:07.176 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.38165.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-09-12T14:57:07.176 INFO:teuthology.orchestra.run.smithi187.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.38165.log.gz 2023-09-12T14:57:07.177 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.38189.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-09-12T14:57:07.177 INFO:teuthology.orchestra.run.smithi187.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38189.log.gz 2023-09-12T14:57:07.177 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44171.log 2023-09-12T14:57:07.177 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43886.log 2023-09-12T14:57:07.178 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.44171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38045.log 2023-09-12T14:57:07.178 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.43886.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-09-12T14:57:07.178 INFO:teuthology.orchestra.run.smithi187.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.1.44171.log.gz 2023-09-12T14:57:07.179 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.38045.log: 90.0%gzip -- replaced with /var/log/ceph/ceph-client.1.43886.log.gz 2023-09-12T14:57:07.179 INFO:teuthology.orchestra.run.smithi187.stderr: -5 --verbose -- /var/log/ceph/ceph-client.3.43998.log 2023-09-12T14:57:07.179 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.5.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.38045.log.gz 2023-09-12T14:57:07.179 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-09-12T14:57:07.180 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.3.43998.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-09-12T14:57:07.180 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38213.log 2023-09-12T14:57:07.180 INFO:teuthology.orchestra.run.smithi187.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.3.43998.log.gz 2023-09-12T14:57:07.181 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38069.log 2023-09-12T14:57:07.181 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.38213.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 86.6% 2023-09-12T14:57:07.181 INFO:teuthology.orchestra.run.smithi187.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38213.log.gz 2023-09-12T14:57:07.181 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.38069.log: gzip 86.6% -- replaced with /var/log/ceph/ceph-client.admin.38069.log.gz -5 2023-09-12T14:57:07.182 INFO:teuthology.orchestra.run.smithi187.stderr: --verbose -- /var/log/ceph/ceph.audit.log 2023-09-12T14:57:07.182 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mon.c.log: 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2023-09-12T14:57:07.182 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-09-12T14:57:07.183 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-09-12T14:57:07.183 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-09-12T14:57:07.183 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mds.b.log: 92.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-09-12T14:57:07.183 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38141.log 2023-09-12T14:57:07.184 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mon.b.log: 91.9% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-09-12T14:57:07.184 INFO:teuthology.orchestra.run.smithi187.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-09-12T14:57:07.184 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-09-12T14:57:07.185 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38117.log 2023-09-12T14:57:07.185 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.38141.log: /var/log/ceph/ceph-osd.4.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.38141.log.gz 2023-09-12T14:57:07.185 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38093.log 2023-09-12T14:57:07.185 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.38117.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.38117.log.gz 2023-09-12T14:57:07.186 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.38093.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38093.log.gz 2023-09-12T14:57:07.195 INFO:teuthology.orchestra.run.smithi187.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-09-12T14:57:07.210 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45510.log.gz 2023-09-12T14:57:07.211 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48269.log 2023-09-12T14:57:07.211 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46347.log.gz 2023-09-12T14:57:07.211 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48445.log 2023-09-12T14:57:07.211 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45536.log.gz 2023-09-12T14:57:07.212 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44383.log 2023-09-12T14:57:07.212 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48269.log.gz 2023-09-12T14:57:07.212 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47376.log 2023-09-12T14:57:07.213 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48445.log.gz 2023-09-12T14:57:07.213 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48417.log 2023-09-12T14:57:07.213 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44383.log.gz 2023-09-12T14:57:07.214 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38027.log 2023-09-12T14:57:07.214 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47376.log.gz 2023-09-12T14:57:07.214 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48685.log 2023-09-12T14:57:07.214 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48417.log.gz 2023-09-12T14:57:07.215 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.46163.log 2023-09-12T14:57:07.215 INFO:teuthology.orchestra.run.smithi131.stderr: 76.9% -- replaced with /var/log/ceph/ceph-client.admin.38027.log.gz 2023-09-12T14:57:07.215 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47696.log 2023-09-12T14:57:07.216 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48685.log.gz 2023-09-12T14:57:07.216 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.2.46163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44331.log 2023-09-12T14:57:07.216 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44730.log 2023-09-12T14:57:07.216 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47696.log.gz 2023-09-12T14:57:07.217 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44331.log: 91.2% -- replaced with /var/log/ceph/ceph-client.2.46163.log.gz 2023-09-12T14:57:07.217 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45069.log 2023-09-12T14:57:07.217 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49443.log 2023-09-12T14:57:07.217 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44331.log.gz 2023-09-12T14:57:07.218 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44730.log: /var/log/ceph/ceph-client.admin.45069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46876.log 2023-09-12T14:57:07.218 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44730.log.gz 2023-09-12T14:57:07.218 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45069.log.gz 2023-09-12T14:57:07.219 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38088.log 2023-09-12T14:57:07.219 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49443.log.gz 2023-09-12T14:57:07.219 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45614.log 2023-09-12T14:57:07.219 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46876.log.gz 2023-09-12T14:57:07.220 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46980.log 2023-09-12T14:57:07.220 INFO:teuthology.orchestra.run.smithi131.stderr: 75.8% -- replaced with /var/log/ceph/ceph-client.admin.38088.log.gz 2023-09-12T14:57:07.220 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45614.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-09-12T14:57:07.221 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45614.log.gz 2023-09-12T14:57:07.221 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45796.log 2023-09-12T14:57:07.221 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-osd.0.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46980.log.gz 2023-09-12T14:57:07.221 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38272.log 2023-09-12T14:57:07.222 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45067.log 2023-09-12T14:57:07.222 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45796.log.gz 2023-09-12T14:57:07.222 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47085.log 2023-09-12T14:57:07.222 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- 86.6% /var/log/ceph/ceph-client.admin.49360.log 2023-09-12T14:57:07.223 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38272.log.gz 2023-09-12T14:57:07.223 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45067.log: /var/log/ceph/ceph-client.admin.47085.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45067.log.gz 2023-09-12T14:57:07.223 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47535.log 2023-09-12T14:57:07.224 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47085.log.gz 2023-09-12T14:57:07.224 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38277.log 2023-09-12T14:57:07.224 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49360.log: /var/log/ceph/ceph-client.admin.47535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50343.log 2023-09-12T14:57:07.224 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49360.log.gz 2023-09-12T14:57:07.225 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38277.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47535.log.gz 2023-09-12T14:57:07.225 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48497.log 2023-09-12T14:57:07.225 INFO:teuthology.orchestra.run.smithi131.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.38277.log.gz 2023-09-12T14:57:07.226 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47220.log 2023-09-12T14:57:07.226 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50343.log.gz 2023-09-12T14:57:07.226 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38201.log 2023-09-12T14:57:07.226 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47220.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48497.log.gz 2023-09-12T14:57:07.227 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47982.log 2023-09-12T14:57:07.227 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47220.log.gz 2023-09-12T14:57:07.227 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49784.log 2023-09-12T14:57:07.228 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38201.log: /var/log/ceph/ceph-client.admin.47982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48929.log 2023-09-12T14:57:07.228 INFO:teuthology.orchestra.run.smithi131.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.38201.log.gz 2023-09-12T14:57:07.228 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49784.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47982.log.gz 2023-09-12T14:57:07.228 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49007.log 2023-09-12T14:57:07.229 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49784.log.gz 2023-09-12T14:57:07.229 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38165.log 2023-09-12T14:57:07.229 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48929.log.gz 2023-09-12T14:57:07.229 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45070.log 2023-09-12T14:57:07.230 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49007.log: /var/log/ceph/ceph-client.admin.38165.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49007.log.gz 2023-09-12T14:57:07.230 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46928.log 2023-09-12T14:57:07.230 INFO:teuthology.orchestra.run.smithi131.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.38165.log.gz 2023-09-12T14:57:07.231 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49881.log 2023-09-12T14:57:07.231 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45070.log: /var/log/ceph/ceph-client.admin.46928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50029.log 2023-09-12T14:57:07.231 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45070.log.gz 2023-09-12T14:57:07.232 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49881.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46928.log.gz 2023-09-12T14:57:07.232 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50003.log 2023-09-12T14:57:07.232 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49881.log.gz 2023-09-12T14:57:07.232 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45903.log 2023-09-12T14:57:07.233 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50003.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50029.log.gz 2023-09-12T14:57:07.233 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46073.log 2023-09-12T14:57:07.233 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50003.log.gz 2023-09-12T14:57:07.234 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49386.log 2023-09-12T14:57:07.234 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45903.log.gz 2023-09-12T14:57:07.234 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48008.log 2023-09-12T14:57:07.234 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46073.log.gz 2023-09-12T14:57:07.235 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48301.log 2023-09-12T14:57:07.235 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48008.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49386.log.gz 2023-09-12T14:57:07.235 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46609.log 2023-09-12T14:57:07.236 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48301.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48008.log.gz 2023-09-12T14:57:07.236 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46453.log 2023-09-12T14:57:07.236 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48301.log.gz 2023-09-12T14:57:07.236 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46609.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-09-12T14:57:07.237 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46609.log.gz 2023-09-12T14:57:07.237 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46373.log 2023-09-12T14:57:07.237 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46453.log.gz 2023-09-12T14:57:07.238 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44849.log 2023-09-12T14:57:07.238 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50187.log 2023-09-12T14:57:07.238 INFO:teuthology.orchestra.run.smithi131.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46373.log.gz 2023-09-12T14:57:07.238 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46027.log 2023-09-12T14:57:07.239 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44849.log.gz 2023-09-12T14:57:07.239 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45640.log 2023-09-12T14:57:07.239 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50187.log.gz 2023-09-12T14:57:07.240 INFO:teuthology.orchestra.run.smithi131.stderr:gzip/var/log/ceph/ceph-client.0.46027.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49628.log 2023-09-12T14:57:07.240 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46531.log 2023-09-12T14:57:07.240 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45640.log.gz 2023-09-12T14:57:07.240 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48391.log 2023-09-12T14:57:07.241 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49628.log: /var/log/ceph/ceph-client.admin.46531.log: 90.9% -- replaced with /var/log/ceph/ceph-client.0.46027.log.gz 2023-09-12T14:57:07.241 INFO:teuthology.orchestra.run.smithi131.stderr:gzip 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49628.log.gz 2023-09-12T14:57:07.241 INFO:teuthology.orchestra.run.smithi131.stderr: -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-09-12T14:57:07.242 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46531.log.gz 2023-09-12T14:57:07.242 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45266.log 2023-09-12T14:57:07.242 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48391.log.gz 2023-09-12T14:57:07.242 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44756.log 2023-09-12T14:57:07.243 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48877.log 2023-09-12T14:57:07.243 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45266.log.gz 2023-09-12T14:57:07.243 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49135.log 2023-09-12T14:57:07.244 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48877.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44756.log.gz 2023-09-12T14:57:07.244 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47800.log 2023-09-12T14:57:07.244 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48877.log.gz 2023-09-12T14:57:07.244 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49602.log 2023-09-12T14:57:07.245 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49135.log.gz 2023-09-12T14:57:07.245 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47059.log 2023-09-12T14:57:07.245 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49602.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47800.log.gz 2023-09-12T14:57:07.246 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45770.log 2023-09-12T14:57:07.246 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49602.log.gz 2023-09-12T14:57:07.246 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49576.log 2023-09-12T14:57:07.246 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47059.log.gz 2023-09-12T14:57:07.247 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47404.log 2023-09-12T14:57:07.247 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45770.log.gz 2023-09-12T14:57:07.247 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38282.log 2023-09-12T14:57:07.248 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45744.log 2023-09-12T14:57:07.248 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49576.log.gz 2023-09-12T14:57:07.248 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47404.log.gz 2023-09-12T14:57:07.248 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50525.log 2023-09-12T14:57:07.249 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45744.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.38282.log.gz 2023-09-12T14:57:07.249 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47246.log 2023-09-12T14:57:07.249 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45744.log.gz 2023-09-12T14:57:07.250 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49308.log 2023-09-12T14:57:07.250 INFO:teuthology.orchestra.run.smithi131.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.50525.log.gz 2023-09-12T14:57:07.250 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44571.log 2023-09-12T14:57:07.251 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49308.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47246.log.gz 2023-09-12T14:57:07.251 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44493.log 2023-09-12T14:57:07.251 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49308.log.gz 2023-09-12T14:57:07.251 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45057.log 2023-09-12T14:57:07.252 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44571.log.gz 2023-09-12T14:57:07.252 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46687.log 2023-09-12T14:57:07.252 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45057.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44493.log.gz 2023-09-12T14:57:07.253 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49810.log 2023-09-12T14:57:07.253 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45057.log.gz 2023-09-12T14:57:07.253 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47904.log 2023-09-12T14:57:07.254 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46687.log.gz 2023-09-12T14:57:07.254 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49241.log 2023-09-12T14:57:07.254 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47904.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49810.log.gz 2023-09-12T14:57:07.254 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45064.log 2023-09-12T14:57:07.255 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47904.log.gz 2023-09-12T14:57:07.255 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48138.log 2023-09-12T14:57:07.255 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49241.log.gz 2023-09-12T14:57:07.255 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48799.log 2023-09-12T14:57:07.256 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48138.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45064.log.gz 2023-09-12T14:57:07.256 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49758.log 2023-09-12T14:57:07.256 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48138.log.gz 2023-09-12T14:57:07.257 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46321.log 2023-09-12T14:57:07.257 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48799.log.gz 2023-09-12T14:57:07.257 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48903.log 2023-09-12T14:57:07.257 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46321.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49758.log.gz 2023-09-12T14:57:07.258 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46713.log 2023-09-12T14:57:07.258 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46321.log.gz 2023-09-12T14:57:07.258 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45588.log 2023-09-12T14:57:07.259 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46661.log 2023-09-12T14:57:07.259 INFO:teuthology.orchestra.run.smithi131.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48903.log.gz 2023-09-12T14:57:07.259 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49550.log 2023-09-12T14:57:07.259 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46713.log.gz 2023-09-12T14:57:07.260 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46661.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45588.log.gz 2023-09-12T14:57:07.260 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44409.log 2023-09-12T14:57:07.260 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44853.log 2023-09-12T14:57:07.261 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46661.log.gz 2023-09-12T14:57:07.261 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49550.log: /var/log/ceph/ceph-client.admin.44409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45822.log 2023-09-12T14:57:07.261 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49550.log.gz 2023-09-12T14:57:07.261 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44409.log.gz 2023-09-12T14:57:07.262 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50499.log 2023-09-12T14:57:07.262 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45822.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44853.log.gz 2023-09-12T14:57:07.262 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38221.log 2023-09-12T14:57:07.263 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45822.log.gz 2023-09-12T14:57:07.263 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48627.log 2023-09-12T14:57:07.263 INFO:teuthology.orchestra.run.smithi131.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.50499.log.gz 2023-09-12T14:57:07.264 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46583.log 2023-09-12T14:57:07.264 INFO:teuthology.orchestra.run.smithi131.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.38221.log.gz 2023-09-12T14:57:07.264 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49907.log 2023-09-12T14:57:07.265 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48627.log.gz 2023-09-12T14:57:07.265 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46583.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-09-12T14:57:07.265 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38100.log 2023-09-12T14:57:07.265 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46583.log.gz 2023-09-12T14:57:07.266 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44213.log 2023-09-12T14:57:07.266 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49907.log.gz 2023-09-12T14:57:07.266 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38100.log: 76.0% -- replaced with /var/log/ceph/ceph-client.admin.38100.log.gz 2023-09-12T14:57:07.267 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48471.log 2023-09-12T14:57:07.267 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47430.log 2023-09-12T14:57:07.267 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44213.log.gz 2023-09-12T14:57:07.267 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46211.log 2023-09-12T14:57:07.268 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48471.log.gz 2023-09-12T14:57:07.268 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38249.log 2023-09-12T14:57:07.268 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47430.log: /var/log/ceph/ceph-client.admin.46211.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47430.log.gz 2023-09-12T14:57:07.269 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46211.log.gz 2023-09-12T14:57:07.269 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45458.log 2023-09-12T14:57:07.269 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48655.log 2023-09-12T14:57:07.269 INFO:teuthology.orchestra.run.smithi131.stderr: 86.5%/var/log/ceph/ceph-client.admin.45458.log: -- replaced with /var/log/ceph/ceph-client.admin.38249.log.gz 2023-09-12T14:57:07.270 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47826.log 2023-09-12T14:57:07.270 INFO:teuthology.orchestra.run.smithi131.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45458.log.gz 2023-09-12T14:57:07.270 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48655.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48655.log.gz 2023-09-12T14:57:07.271 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45278.log 2023-09-12T14:57:07.271 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50239.log 2023-09-12T14:57:07.271 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46505.log 2023-09-12T14:57:07.272 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47826.log.gz 2023-09-12T14:57:07.272 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45278.log.gz 2023-09-12T14:57:07.272 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50239.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50239.log.gz 2023-09-12T14:57:07.272 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45562.log 2023-09-12T14:57:07.273 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47670.log 2023-09-12T14:57:07.273 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46505.log.gz/var/log/ceph/ceph-client.admin.45562.log: 2023-09-12T14:57:07.273 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50081.log 2023-09-12T14:57:07.274 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45562.log.gz 2023-09-12T14:57:07.274 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47670.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47670.log.gz 2023-09-12T14:57:07.274 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50055.log 2023-09-12T14:57:07.274 INFO:teuthology.orchestra.run.smithi131.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-09-12T14:57:07.275 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50081.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-09-12T14:57:07.275 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45666.log 2023-09-12T14:57:07.275 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50055.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50081.log.gz 2023-09-12T14:57:07.276 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-osd.1.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50055.log.gz 2023-09-12T14:57:07.276 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46269.log 2023-09-12T14:57:07.276 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50213.log 2023-09-12T14:57:07.276 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45666.log: /var/log/ceph/ceph-client.admin.46269.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37985.log 2023-09-12T14:57:07.277 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45666.log.gz 2023-09-12T14:57:07.277 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46269.log.gz 2023-09-12T14:57:07.277 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50213.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50213.log.gz 2023-09-12T14:57:07.278 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50552.log 2023-09-12T14:57:07.278 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49844.log 2023-09-12T14:57:07.278 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37985.log.gz 2023-09-12T14:57:07.278 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48745.log 2023-09-12T14:57:07.279 INFO:teuthology.orchestra.run.smithi131.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50552.log.gz 2023-09-12T14:57:07.279 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48164.log 2023-09-12T14:57:07.279 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49844.log: /var/log/ceph/ceph-client.admin.48745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48719.log 2023-09-12T14:57:07.279 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48745.log.gz 2023-09-12T14:57:07.280 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49844.log.gz 2023-09-12T14:57:07.280 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49959.log 2023-09-12T14:57:07.280 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48164.log.gz 2023-09-12T14:57:07.281 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48601.log 2023-09-12T14:57:07.281 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48719.log: /var/log/ceph/ceph-client.admin.49959.log: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47168.log 2023-09-12T14:57:07.281 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48719.log.gz 2023-09-12T14:57:07.281 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49959.log.gz 2023-09-12T14:57:07.282 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48601.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48601.log.gz 2023-09-12T14:57:07.282 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50602.log 2023-09-12T14:57:07.282 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48190.log 2023-09-12T14:57:07.283 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47168.log.gz/var/log/ceph/ceph-client.admin.50602.log: 2023-09-12T14:57:07.283 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49521.log 2023-09-12T14:57:07.283 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50602.log.gz 2023-09-12T14:57:07.283 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49189.log 2023-09-12T14:57:07.284 INFO:teuthology.orchestra.run.smithi131.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.48190.log.gz 2023-09-12T14:57:07.284 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48771.log 2023-09-12T14:57:07.284 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6%/var/log/ceph/ceph-client.0.49189.log: -- replaced with /var/log/ceph/ceph-client.admin.49521.log.gz 2023-09-12T14:57:07.285 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47033.log 2023-09-12T14:57:07.285 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48771.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48771.log.gz 2023-09-12T14:57:07.285 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47930.log 2023-09-12T14:57:07.285 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48216.log 2023-09-12T14:57:07.286 INFO:teuthology.orchestra.run.smithi131.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.49189.log.gz 2023-09-12T14:57:07.286 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47033.log.gz 2023-09-12T14:57:07.286 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47930.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47930.log.gz 2023-09-12T14:57:07.287 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44782.log 2023-09-12T14:57:07.287 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45271.log 2023-09-12T14:57:07.287 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48216.log.gz 2023-09-12T14:57:07.287 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46954.log 2023-09-12T14:57:07.288 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44782.log.gz/var/log/ceph/ceph-client.admin.45271.log: 2023-09-12T14:57:07.288 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-09-12T14:57:07.288 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45271.log.gz 2023-09-12T14:57:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46954.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46954.log.gz 2023-09-12T14:57:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47748.log 2023-09-12T14:57:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46479.log 2023-09-12T14:57:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49033.log 2023-09-12T14:57:07.290 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% 85.3% -- replaced with /var/log/ceph/ceph.log.gz 2023-09-12T14:57:07.290 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47748.log.gz 2023-09-12T14:57:07.290 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46479.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46479.log.gz 2023-09-12T14:57:07.291 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50135.log 2023-09-12T14:57:07.291 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45692.log 2023-09-12T14:57:07.291 INFO:teuthology.orchestra.run.smithi131.stderr: 86.9%/var/log/ceph/ceph-client.admin.50135.log: -- replaced with /var/log/ceph/ceph-client.admin.49033.log.gz 2023-09-12T14:57:07.291 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46817.log 2023-09-12T14:57:07.292 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44862.log 2023-09-12T14:57:07.292 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50135.log.gz 2023-09-12T14:57:07.292 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45692.log: /var/log/ceph/ceph-client.admin.46817.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45692.log.gz 2023-09-12T14:57:07.293 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46817.log.gz 2023-09-12T14:57:07.293 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50317.log 2023-09-12T14:57:07.294 INFO:teuthology.orchestra.run.smithi187.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-09-12T14:57:07.295 INFO:teuthology.orchestra.run.smithi187.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-09-12T14:57:07.295 INFO:teuthology.orchestra.run.smithi187.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-09-12T14:57:07.300 INFO:teuthology.orchestra.run.smithi187.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-09-12T14:57:07.307 INFO:teuthology.orchestra.run.smithi187.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-09-12T14:57:07.332 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44675.log 2023-09-12T14:57:07.333 INFO:teuthology.orchestra.run.smithi131.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44862.log.gz/var/log/ceph/ceph-client.admin.50317.log: 2023-09-12T14:57:07.333 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49085.log 2023-09-12T14:57:07.334 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50317.log.gz 2023-09-12T14:57:07.334 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44675.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44675.log.gz 2023-09-12T14:57:07.334 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45250.log 2023-09-12T14:57:07.335 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48365.log 2023-09-12T14:57:07.335 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45272.log 2023-09-12T14:57:07.335 INFO:teuthology.orchestra.run.smithi131.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.49085.log.gz 2023-09-12T14:57:07.336 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45250.log.gz 2023-09-12T14:57:07.336 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48365.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48365.log.gz 2023-09-12T14:57:07.336 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44855.log 2023-09-12T14:57:07.336 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46635.log 2023-09-12T14:57:07.337 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5%/var/log/ceph/ceph-client.admin.44855.log: -- replaced with /var/log/ceph/ceph-client.admin.45272.log.gz 2023-09-12T14:57:07.337 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45276.log 2023-09-12T14:57:07.337 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44623.log 2023-09-12T14:57:07.338 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44855.log.gz 2023-09-12T14:57:07.338 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46635.log: /var/log/ceph/ceph-client.admin.45276.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46635.log.gz 2023-09-12T14:57:07.338 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45276.log.gz 2023-09-12T14:57:07.339 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49059.log 2023-09-12T14:57:07.339 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47194.log 2023-09-12T14:57:07.339 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44623.log.gz/var/log/ceph/ceph-client.admin.49059.log: 2023-09-12T14:57:07.339 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47324.log 2023-09-12T14:57:07.340 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49059.log.gz 2023-09-12T14:57:07.340 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47194.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47194.log.gz 2023-09-12T14:57:07.340 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45848.log 2023-09-12T14:57:07.341 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38025.log 2023-09-12T14:57:07.341 INFO:teuthology.orchestra.run.smithi131.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47324.log.gz/var/log/ceph/ceph-client.admin.45848.log: 2023-09-12T14:57:07.341 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47007.log 2023-09-12T14:57:07.341 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38025.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38025.log.gz 2023-09-12T14:57:07.342 INFO:teuthology.orchestra.run.smithi131.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.45848.log.gz 2023-09-12T14:57:07.342 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47587.log 2023-09-12T14:57:07.342 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38736.log 2023-09-12T14:57:07.343 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47007.log.gz/var/log/ceph/ceph-client.admin.47587.log: 2023-09-12T14:57:07.343 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38611.log 2023-09-12T14:57:07.343 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47587.log.gz 2023-09-12T14:57:07.343 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38204.log 2023-09-12T14:57:07.344 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38736.log: /var/log/ceph/ceph-client.admin.38611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47272.log 2023-09-12T14:57:07.344 INFO:teuthology.orchestra.run.smithi131.stderr: 86.8%/var/log/ceph/ceph-client.admin.38204.log: -- replaced with /var/log/ceph/ceph-client.admin.38611.log.gz 2023-09-12T14:57:07.344 INFO:teuthology.orchestra.run.smithi131.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.38736.log.gz 2023-09-12T14:57:07.345 INFO:teuthology.orchestra.run.smithi131.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.38204.log.gz 2023-09-12T14:57:07.345 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49282.log 2023-09-12T14:57:07.345 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45718.log 2023-09-12T14:57:07.346 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47272.log.gz/var/log/ceph/ceph-client.admin.49282.log: 2023-09-12T14:57:07.346 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46237.log 2023-09-12T14:57:07.346 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49282.log.gz 2023-09-12T14:57:07.346 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45718.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45718.log.gz 2023-09-12T14:57:07.347 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45484.log 2023-09-12T14:57:07.347 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46399.log 2023-09-12T14:57:07.347 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46237.log.gz/var/log/ceph/ceph-client.admin.45484.log: 2023-09-12T14:57:07.348 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49334.log 2023-09-12T14:57:07.348 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45484.log.gz 2023-09-12T14:57:07.348 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38051.log 2023-09-12T14:57:07.349 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46399.log: /var/log/ceph/ceph-client.admin.49334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48523.log 2023-09-12T14:57:07.349 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0%/var/log/ceph/ceph-client.admin.38051.log: -- replaced with /var/log/ceph/ceph-client.admin.49334.log.gz 2023-09-12T14:57:07.349 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46399.log.gz 2023-09-12T14:57:07.349 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38051.log.gz 2023-09-12T14:57:07.350 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48549.log 2023-09-12T14:57:07.350 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50395.log 2023-09-12T14:57:07.350 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4%/var/log/ceph/ceph-client.admin.48549.log: -- replaced with /var/log/ceph/ceph-client.admin.48523.log.gz 2023-09-12T14:57:07.351 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38063.log 2023-09-12T14:57:07.351 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48549.log.gzgzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.33136.log 2023-09-12T14:57:07.351 INFO:teuthology.orchestra.run.smithi131.stderr: 2023-09-12T14:57:07.352 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50395.log: /var/log/ceph/ceph-client.admin.38063.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38063.log.gz 2023-09-12T14:57:07.352 INFO:teuthology.orchestra.run.smithi131.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50395.log.gz 2023-09-12T14:57:07.352 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46902.log 2023-09-12T14:57:07.352 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph.tmp-client.admin.33136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47482.log 2023-09-12T14:57:07.353 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.33136.log.gz 2023-09-12T14:57:07.353 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46902.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-09-12T14:57:07.353 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46902.log.gz 2023-09-12T14:57:07.354 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47482.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-09-12T14:57:07.354 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5%/var/log/ceph/ceph.audit.log: -- replaced with /var/log/ceph/ceph-client.admin.47482.log.gz 2023-09-12T14:57:07.354 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46425.log 2023-09-12T14:57:07.355 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-mds.c.log: 91.8% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-09-12T14:57:07.355 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-09-12T14:57:07.355 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47508.log 2023-09-12T14:57:07.355 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46425.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-09-12T14:57:07.356 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46425.log.gz/var/log/ceph/ceph-client.admin.47508.log: 2023-09-12T14:57:07.356 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47350.log 2023-09-12T14:57:07.356 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47508.log.gz 2023-09-12T14:57:07.357 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44112.log 2023-09-12T14:57:07.357 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49706.log 2023-09-12T14:57:07.357 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47350.log.gz 2023-09-12T14:57:07.357 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44112.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44112.log.gz 2023-09-12T14:57:07.358 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44808.log 2023-09-12T14:57:07.358 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50447.log 2023-09-12T14:57:07.358 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49706.log: /var/log/ceph/ceph-client.admin.44808.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49706.log.gz 2023-09-12T14:57:07.359 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44808.log.gz 2023-09-12T14:57:07.359 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38039.log 2023-09-12T14:57:07.359 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46295.log 2023-09-12T14:57:07.359 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50447.log: /var/log/ceph/ceph-client.admin.38039.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38039.log.gz 2023-09-12T14:57:07.360 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50447.log.gz 2023-09-12T14:57:07.360 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44545.log 2023-09-12T14:57:07.360 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47774.log 2023-09-12T14:57:07.361 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46295.log: /var/log/ceph/ceph-client.admin.44545.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46295.log.gz 2023-09-12T14:57:07.361 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44545.log.gz 2023-09-12T14:57:07.361 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44861.log 2023-09-12T14:57:07.361 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44704.log 2023-09-12T14:57:07.362 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47774.log: /var/log/ceph/ceph-client.admin.44861.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47774.log.gz 2023-09-12T14:57:07.362 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44861.log.gz 2023-09-12T14:57:07.362 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46843.log 2023-09-12T14:57:07.363 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48034.log 2023-09-12T14:57:07.363 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44704.log: /var/log/ceph/ceph-client.admin.46843.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44704.log.gz 2023-09-12T14:57:07.363 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46843.log.gz 2023-09-12T14:57:07.363 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50265.log 2023-09-12T14:57:07.364 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46765.log 2023-09-12T14:57:07.364 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48034.log: /var/log/ceph/ceph-client.admin.50265.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48034.log.gz 2023-09-12T14:57:07.364 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50265.log.gz 2023-09-12T14:57:07.365 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50369.log 2023-09-12T14:57:07.365 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46765.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46765.log.gz 2023-09-12T14:57:07.365 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46791.log 2023-09-12T14:57:07.365 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49933.log 2023-09-12T14:57:07.366 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50369.log: /var/log/ceph/ceph-client.admin.46791.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50369.log.gz 2023-09-12T14:57:07.366 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46791.log.gz 2023-09-12T14:57:07.366 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48243.log 2023-09-12T14:57:07.367 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50161.log 2023-09-12T14:57:07.367 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49933.log: /var/log/ceph/ceph-client.admin.48243.log: 87.7% 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49933.log.gz 2023-09-12T14:57:07.367 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48243.log.gz 2023-09-12T14:57:07.368 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50576.log 2023-09-12T14:57:07.368 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45068.log 2023-09-12T14:57:07.368 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50161.log: /var/log/ceph/ceph-client.admin.50576.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50161.log.gz 2023-09-12T14:57:07.368 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50576.log.gz 2023-09-12T14:57:07.369 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47561.log 2023-09-12T14:57:07.369 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44597.log 2023-09-12T14:57:07.369 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45068.log: /var/log/ceph/ceph-client.admin.47561.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45068.log.gz 2023-09-12T14:57:07.370 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47561.log.gz 2023-09-12T14:57:07.370 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45275.log 2023-09-12T14:57:07.370 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49732.log 2023-09-12T14:57:07.370 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44597.log: /var/log/ceph/ceph-client.admin.45275.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44597.log.gz 2023-09-12T14:57:07.371 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45275.log.gz 2023-09-12T14:57:07.371 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47956.log 2023-09-12T14:57:07.371 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49417.log 2023-09-12T14:57:07.371 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49732.log: /var/log/ceph/ceph-client.admin.47956.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49732.log.gz 2023-09-12T14:57:07.372 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47956.log.gz 2023-09-12T14:57:07.372 INFO:teuthology.orchestra.run.smithi131.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-09-12T14:57:07.372 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49680.log 2023-09-12T14:57:07.373 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47722.log 2023-09-12T14:57:07.373 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49417.log: /var/log/ceph/ceph-client.admin.49680.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44860.log 2023-09-12T14:57:07.373 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49417.log.gz 2023-09-12T14:57:07.374 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49680.log.gz 2023-09-12T14:57:07.374 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47722.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47722.log.gz 2023-09-12T14:57:07.374 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48851.log 2023-09-12T14:57:07.374 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47613.log 2023-09-12T14:57:07.375 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48851.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44860.log.gz 2023-09-12T14:57:07.375 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38075.log 2023-09-12T14:57:07.375 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50473.log 2023-09-12T14:57:07.376 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48851.log.gz 2023-09-12T14:57:07.376 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47613.log: /var/log/ceph/ceph-client.admin.38075.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38075.log.gz 2023-09-12T14:57:07.376 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47613.log.gz 2023-09-12T14:57:07.376 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38238.log 2023-09-12T14:57:07.377 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45066.log 2023-09-12T14:57:07.377 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4%/var/log/ceph/ceph-client.admin.38238.log: -- replaced with /var/log/ceph/ceph-client.admin.50473.log.gz 2023-09-12T14:57:07.377 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44649.log 2023-09-12T14:57:07.378 INFO:teuthology.orchestra.run.smithi131.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.38238.log.gz 2023-09-12T14:57:07.378 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44465.log 2023-09-12T14:57:07.378 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45066.log: /var/log/ceph/ceph-client.admin.44649.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45066.log.gz 2023-09-12T14:57:07.379 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44649.log.gz 2023-09-12T14:57:07.379 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44278.log 2023-09-12T14:57:07.379 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47639.log 2023-09-12T14:57:07.379 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44465.log: /var/log/ceph/ceph-client.admin.44278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48825.log 2023-09-12T14:57:07.380 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44465.log.gz 2023-09-12T14:57:07.380 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5%/var/log/ceph/ceph-client.admin.47639.log: -- replaced with /var/log/ceph/ceph-client.admin.44278.log.gz 2023-09-12T14:57:07.380 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50109.log 2023-09-12T14:57:07.381 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47639.log.gz 2023-09-12T14:57:07.381 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50421.log 2023-09-12T14:57:07.381 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48825.log: /var/log/ceph/ceph-client.admin.50109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48575.log 2023-09-12T14:57:07.381 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50421.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48825.log.gz 2023-09-12T14:57:07.382 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50109.log.gz 2023-09-12T14:57:07.382 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38273.log 2023-09-12T14:57:07.382 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50421.log.gz 2023-09-12T14:57:07.382 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44180.log 2023-09-12T14:57:07.383 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48575.log: /var/log/ceph/ceph-client.admin.38273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49654.log 2023-09-12T14:57:07.383 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9%/var/log/ceph/ceph-client.admin.44180.log: -- replaced with /var/log/ceph/ceph-client.admin.48575.log.gz 2023-09-12T14:57:07.383 INFO:teuthology.orchestra.run.smithi131.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.38273.log.gz 2023-09-12T14:57:07.384 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44180.log.gz 2023-09-12T14:57:07.384 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48981.log 2023-09-12T14:57:07.384 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48955.log 2023-09-12T14:57:07.384 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49654.log: /var/log/ceph/ceph-client.admin.48981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48112.log 2023-09-12T14:57:07.385 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48955.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49654.log.gz 2023-09-12T14:57:07.385 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48981.log.gz 2023-09-12T14:57:07.385 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48955.log.gz 2023-09-12T14:57:07.386 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49469.log 2023-09-12T14:57:07.386 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47456.log 2023-09-12T14:57:07.386 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48112.log: /var/log/ceph/ceph-client.admin.49469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46557.log 2023-09-12T14:57:07.387 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47456.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48112.log.gz 2023-09-12T14:57:07.387 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49469.log.gz 2023-09-12T14:57:07.387 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44304.log 2023-09-12T14:57:07.387 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47456.log.gz 2023-09-12T14:57:07.388 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46739.log 2023-09-12T14:57:07.388 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46557.log: /var/log/ceph/ceph-client.admin.44304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45063.log 2023-09-12T14:57:07.388 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46557.log.gz 2023-09-12T14:57:07.388 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46739.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44304.log.gz 2023-09-12T14:57:07.389 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46739.log.gz 2023-09-12T14:57:07.389 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44437.log 2023-09-12T14:57:07.389 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48060.log 2023-09-12T14:57:07.390 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45063.log: /var/log/ceph/ceph-client.admin.44437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44357.log 2023-09-12T14:57:07.390 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5%/var/log/ceph/ceph-client.admin.48060.log: -- replaced with /var/log/ceph/ceph-client.admin.45063.log.gz 2023-09-12T14:57:07.390 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44437.log.gz 2023-09-12T14:57:07.391 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48060.log.gz 2023-09-12T14:57:07.391 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47852.log 2023-09-12T14:57:07.391 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48086.log 2023-09-12T14:57:07.391 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44357.log: /var/log/ceph/ceph-client.admin.47852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44519.log 2023-09-12T14:57:07.392 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44357.log.gz 2023-09-12T14:57:07.392 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48086.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47852.log.gz 2023-09-12T14:57:07.392 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48086.log.gz 2023-09-12T14:57:07.393 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44859.log 2023-09-12T14:57:07.393 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44519.log: /var/log/ceph/ceph-client.admin.44859.log: 93.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-09-12T14:57:07.393 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44519.log.gz 2023-09-12T14:57:07.393 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44859.log.gz 2023-09-12T14:57:07.394 INFO:teuthology.orchestra.run.smithi131.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-09-12T14:57:07.394 INFO:teuthology.orchestra.run.smithi131.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-09-12T14:57:07.394 INFO:teuthology.orchestra.run.smithi131.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-09-12T14:57:07.396 INFO:teuthology.orchestra.run.smithi131.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-09-12T14:57:07.397 INFO:teuthology.orchestra.run.smithi131.stderr: 2023-09-12T14:57:07.397 INFO:teuthology.orchestra.run.smithi131.stderr:real 0m0.278s 2023-09-12T14:57:07.397 INFO:teuthology.orchestra.run.smithi131.stderr:user 0m0.982s 2023-09-12T14:57:07.397 INFO:teuthology.orchestra.run.smithi131.stderr:sys 0m0.134s 2023-09-12T14:57:07.398 INFO:teuthology.orchestra.run.smithi187.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-09-12T14:57:07.398 INFO:teuthology.orchestra.run.smithi187.stderr: 2023-09-12T14:57:07.398 INFO:teuthology.orchestra.run.smithi187.stderr:real 0m0.180s 2023-09-12T14:57:07.399 INFO:teuthology.orchestra.run.smithi187.stderr:user 0m0.723s 2023-09-12T14:57:07.399 INFO:teuthology.orchestra.run.smithi187.stderr:sys 0m0.051s 2023-09-12T14:57:07.399 INFO:tasks.ceph:Archiving logs... 2023-09-12T14:57:07.400 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395049/remote/smithi131/log 2023-09-12T14:57:07.401 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-12T14:57:07.728 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395049/remote/smithi187/log 2023-09-12T14:57:07.730 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-12T14:57:07.818 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-09-12T14:57:07.839 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-09-12T14:57:07.840 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-12T14:57:07.843 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-12T14:57:07.916 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-09-12T14:57:07.925 INFO:teuthology.task.clock:Checking final clock skew... 2023-09-12T14:57:07.926 DEBUG:teuthology.orchestra.run.smithi131:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-12T14:57:07.929 DEBUG:teuthology.orchestra.run.smithi187:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-12T14:57:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: remote refid st t when poll reach delay offset jitter 2023-09-12T14:57:07.940 INFO:teuthology.orchestra.run.smithi131.stdout:============================================================================== 2023-09-12T14:57:07.940 INFO:teuthology.orchestra.run.smithi131.stdout:+hv01.front.sepi 44.4.53.2 2 u 12 64 77 0.077 -2.188 2.983 2023-09-12T14:57:07.940 INFO:teuthology.orchestra.run.smithi131.stdout:*hv02.front.sepi 63.231.80.2 3 u 17 64 77 0.096 +0.064 1.255 2023-09-12T14:57:07.941 INFO:teuthology.orchestra.run.smithi131.stdout:+hv03.front.sepi 74.6.168.72 3 u 25 64 77 0.088 -5.429 3.787 2023-09-12T14:57:07.941 INFO:teuthology.orchestra.run.smithi131.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-09-12T14:57:07.942 INFO:teuthology.orchestra.run.smithi187.stdout: remote refid st t when poll reach delay offset jitter 2023-09-12T14:57:07.943 INFO:teuthology.orchestra.run.smithi187.stdout:============================================================================== 2023-09-12T14:57:07.943 INFO:teuthology.orchestra.run.smithi187.stdout:+hv01.front.sepi 44.4.53.2 2 u 22 64 77 0.077 +0.142 1.102 2023-09-12T14:57:07.943 INFO:teuthology.orchestra.run.smithi187.stdout:*hv02.front.sepi 63.231.80.2 3 u 19 64 77 0.109 -0.332 1.273 2023-09-12T14:57:07.943 INFO:teuthology.orchestra.run.smithi187.stdout:+hv03.front.sepi 74.6.168.72 3 u 18 64 77 0.105 -2.389 1.310 2023-09-12T14:57:07.944 INFO:teuthology.orchestra.run.smithi187.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-09-12T14:57:07.945 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-09-12T14:57:07.954 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-09-12T14:57:07.955 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-09-12T14:57:07.964 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-09-12T14:57:07.973 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-09-12T14:57:07.983 INFO:teuthology.task.internal:Duration was 814.777998 seconds 2023-09-12T14:57:07.983 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-09-12T14:57:07.992 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-09-12T14:57:07.993 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-12T14:57:07.995 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-12T14:57:08.026 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2023-09-12T14:57:08.053 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-09-12T14:57:08.053 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi131.front.sepia.ceph.com 2023-09-12T14:57:08.053 DEBUG:teuthology.orchestra.run.smithi131:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-12T14:57:08.063 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi187.front.sepia.ceph.com 2023-09-12T14:57:08.064 DEBUG:teuthology.orchestra.run.smithi187:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-12T14:57:08.073 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-09-12T14:57:08.074 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-12T14:57:08.106 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-12T14:57:08.143 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-09-12T14:57:08.143 DEBUG:teuthology.orchestra.run.smithi131:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-12T14:57:08.170 DEBUG:teuthology.orchestra.run.smithi187:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-12T14:57:08.240 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-09-12T14:57:08.251 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-09-12T14:57:08.252 DEBUG:teuthology.orchestra.run.smithi131:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-12T14:57:08.262 DEBUG:teuthology.orchestra.run.smithi187:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-12T14:57:08.275 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-09-12T14:57:08.286 DEBUG:teuthology.orchestra.run.smithi131:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-12T14:57:08.306 DEBUG:teuthology.orchestra.run.smithi187:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-12T14:57:08.315 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = core 2023-09-12T14:57:08.332 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = core 2023-09-12T14:57:08.344 DEBUG:teuthology.orchestra.run.smithi131:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-12T14:57:08.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T14:57:08.372 DEBUG:teuthology.orchestra.run.smithi187:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-12T14:57:08.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-12T14:57:08.390 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-09-12T14:57:08.400 INFO:teuthology.task.internal:Transferring archived files... 2023-09-12T14:57:08.401 DEBUG:teuthology.misc:Transferring archived files from smithi131:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395049/remote/smithi131 2023-09-12T14:57:08.401 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-12T14:57:08.432 DEBUG:teuthology.misc:Transferring archived files from smithi187:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395049/remote/smithi187 2023-09-12T14:57:08.433 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-12T14:57:08.452 INFO:teuthology.task.internal:Removing archive directory... 2023-09-12T14:57:08.452 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-12T14:57:08.474 DEBUG:teuthology.orchestra.run.smithi187:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-12T14:57:08.494 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-09-12T14:57:08.506 INFO:teuthology.task.internal:Not uploading archives. 2023-09-12T14:57:08.506 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-09-12T14:57:08.516 INFO:teuthology.task.internal:Tidying up after the test... 2023-09-12T14:57:08.517 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-12T14:57:08.522 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-12T14:57:08.525 INFO:teuthology.orchestra.run.smithi131.stdout: 655472 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 14:57 /home/ubuntu/cephtest 2023-09-12T14:57:08.542 INFO:teuthology.orchestra.run.smithi187.stdout: 655472 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 14:57 /home/ubuntu/cephtest 2023-09-12T14:57:08.544 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-09-12T14:57:08.555 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-09-12T14:57:08.576 INFO:teuthology.nuke:Checking targets against current locks 2023-09-12T14:57:08.604 DEBUG:teuthology.nuke:shortname: smithi131 2023-09-12T14:57:08.604 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-12T14:57:08.631 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi131.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395049', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-12 14:35:09.858157', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDimC56/hFFPRUSSnJw57ISHBvV1b41xUIVLsFaU/Q5jjFtRty1ECTHJAz1lZsZYADp2Hs6BIyUbsb23UEm5CDs='} 2023-09-12T14:57:08.632 DEBUG:teuthology.nuke:shortname: smithi187 2023-09-12T14:57:08.633 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-12T14:57:08.658 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi187.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395049', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-12 14:35:09.856194', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPlylNRPAXfYR2xFtC39/4ZS32+rbGLFQP0faK3NmYDP8iUE2PbEIiYiJxboXIcK55ZSOLyk2idPQ2YJ9Br2zsM='} 2023-09-12T14:57:08.682 INFO:teuthology.orchestra.console.smithi131:Power off 2023-09-12T14:57:08.683 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-12T14:57:08.706 DEBUG:teuthology.orchestra.console.smithi131:power off output: Chassis Power Control: Down/Off 2023-09-12T14:57:08.707 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-12T14:57:08.722 INFO:teuthology.orchestra.console.smithi187:Power off 2023-09-12T14:57:08.722 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-12T14:57:08.737 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on 2023-09-12T14:57:08.746 DEBUG:teuthology.orchestra.console.smithi187:power off output: Chassis Power Control: Down/Off 2023-09-12T14:57:08.747 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-12T14:57:08.783 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2023-09-12T14:57:12.739 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-12T14:57:12.785 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-12T14:57:12.850 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on 2023-09-12T14:57:12.896 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2023-09-12T14:57:16.854 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-12T14:57:16.897 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-12T14:57:16.965 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on 2023-09-12T14:57:17.007 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is off 2023-09-12T14:57:17.108 INFO:teuthology.orchestra.console.smithi187:Power off completed 2023-09-12T14:57:20.966 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-12T14:57:21.075 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is off 2023-09-12T14:57:21.177 INFO:teuthology.orchestra.console.smithi131:Power off completed 2023-09-12T14:57:21.282 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota} duration: 814.7779984474182 failure_reason: 'Test failure: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)' flavor: default owner: scheduled_pdonnell@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=7f383fe11333408a9de717dc11637562 status: fail success: false 2023-09-12T14:57:21.283 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-12T14:57:21.388 INFO:teuthology.run:FAIL